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SUMMARY 


THE  PROBLEM 

The  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  has  been  tasked  to  develop  a  methodology  for  conducting  training 
program  evaluations  (TPEs)  and  procedures  for  revising  training  programs  to 
eliminate  performance  deficiencies  discovered  during  TPEs.  In  a  previous 
contract,  a  prototype  methodology  for  TPEs  was  developed,  but  that  effort 
did  not  include  the  development  of  procedures  to  correct  flaws  In  training 
programs.  The  purpose  of  the  project  reported  here  was  to  develop  the 
corrective  procedures  for  shortcomings  In  training  that  might  be  discovered 
during  TPEs. 

APPROACH 

The  various  training  shortcomings  and  indications  thereof  identified  in 
the  earlier  study  were  the  focus  during  this  project.  The  framework  for 
viewing  the  problems  Included  two  major  foci.  First,  all  aspects  of 
training  were  viewed  as  an  Integrated  system.  The  original  purpose  was  thus 
expanded  to  include  consideration  of  administrative  issues  related  to 
training  as  well  as  procedures  for  correcting  instructional  deficiencies. 

The  training  analyst  is  thus  provided  perspective  for  viewing  training 
problems  whether  or  not  he  is  in  a  position  to  take  action  regarding  all  of 
them.  Second,  the  methodology  for  identifying  performance  deficiencies 
focused  on  systematic  observations  and  analyses  of  instructional  processes 
and  their  effects  on  trainees'  daily  achievements.  A  list  of  principles 
regarding  the  use  of  practice,  guidance,  and  feedback  during  training  was 
developed  to  guide  both  the  identification  of  instructional  deficiencies  and 
their  correction. 


V 


RESULTS 


Analyses  of  problems  were  grouped  under  two  broad  topics.  Problems 
related  to  Instructional  processes  were  examined  In  terms  of  their  sources 
that  can  be  Identified  by  observations  during  training.  Corrective 
measures  involving  effective  use  of  practice,  guidance,  and  feedback  were 
then  explained  as  applicable  to  each  source  of  difficulty.  The  second 
broad  group  of  problems  concerned  areas  of  administrative  responsibility. 
They  were  addressed  under  five  heads:  Indications  of  Need  to  Redesign 
Training  Programs;  Training  Support  and  Facilitation;  Instructor 
Characteristics  and  Qualifications;  Management  of  Attitudes;  and  Problems 
in  Performance  Testing.  The  focus  for  the  discussion  of  each  of  these  five 
areas  was  the  need  for  administrative  resolve  and  action  to  prevent 
problems  from  arising  when  possible,  and  to  take  prompt  corrective  steps 
when  they  do  occur. 


PREFACE 


This  report  describes  the  results  of  a  research  project  that  was 
undertaken  to  develop  procedures  for  eliminating  training  deficiencies 
discovered  during  training  effectiveness  evaluations.  The  research  was 
sponsored  by  the  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  { ARI )  under  Contract  No.  MDA903-79-C-0545.  Mr.  Donald  M. 
Kristiansen  of  the  ARI  Field  Unit  at  Ft.  Knox,  Kentucky,  was  the 
Contracting  Officer's  Technical  Representative.  Dr.  Paul  W.  Caro  was  the 
Seville  Program  Manager  for  the  project,  while  Mr.  Jeffery  L.  Maxey  served 
as  Project  Director.  Dr.  William  D.  Spears,  Mr.  Marvin  D.  Roush,  II,  and 
Mr.  Robert  N.  Isley  comprised  the  project  staff. 
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I.  INTRODUCTION  AND  APPROACH 


Effective  training  is  the  primary  requisite  for  military  preparedness. 
Ensuring  effective  training  with  a  minimum  investment  of  time,  effort,  and 
resources  is  becoming  more  and  more  difficult  due  to  the  increasing 
complexity  and  diversity  of  weapon  systems.  The  skills  required  to  employ 
modern  weapon  systems  are  often  highly  technical  and  always  exacting.  As  a 
result,  the  design  of  optimal  training  programs  requires  the  same  care  and 
attention  to  detail  that  is  necessary  to  design  the  equipment. 

Current  training  technology  provides  valuable  guidance  for  designing 
military  training  programs  that  are  both  effective  and  efficient.*  One 
shortcoming,  however,  is  that  training  technology  is  as  much  an  art  as  a 
science.  It  is  not  likely  that  an  optimum  program  can  be  designed  on  the 
first  try.  It  will  usually  be  necessary  to  implement  the  program  and  then 
evaluate  its  effectiveness.  In  doing  so,  deficiencies  in  the  training 
program  can  be  identified,  and  the  program  revised  to  eliminate  them. 

Training  program  evaluations  (TPEs)  and  subsequent  program  revisions 
seem  fairly  straightforward,  but  in  practice  they  are  not.  The  Army 
regulations,  manuals,  and  handbooks  on  TPE  and  training  are  explicit 


*For  an  extensive  review  of  literature  related  to  the  systematic  design 
of  training  programs,  see  Montemerlo,  M.D.,  and  Tennyson,  M.E. 

Instructional  Systems  Development:  Conceptual  Analysis  and  Comprehensive 
bibliography  (NAVTRAEQUIPCEN  IH-257).  Orlando,  FL:  Naval  Training 
Equipment  Center,  February,  1976.  For  summaries  of  learning  principles 
applicable  to  military  training,  see  Wheaton,  G.R.,  Rose,  A.M.,  Fingerman, 
P.W.,  Korotkin,  A.L.,  and  Holding,  D.H.  Evaluation  of  the  Effectiveness  of 
Training  Devices:  Literature  Review  and  "Preliminary  Model  (Research 
Memorandum  76-6).  Arlington,  VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  April  1976;  and  Appendix  A  of  Wheaton, 

G.R.,  Fingerman,  P.W.,  Rose,  A.M.,  and  Leonard,  R.L.,  Jr.  Evaluation  of 
the  Effectiveness  of  Training  Devices:  Elaboration  and  Application  of  the 
Predictive  Model  (Research  Memorandum  76-16).  Arlington,  VA:  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences,  July  1976. 
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regarding  what  should  be  done,  but  they  are  not  explicit  on  how  to  do  it. 
This  lack  of  direction  is  a  serious  shortcoming  because  Army  training  ana¬ 
lysts  often  are  not  expert  in  training  technology.  They  need  specific 
guidance  that  explains  how  to  go  about  their  task,  what  to  look  for,  and 
what  to  do  about  the  training  deficiencies  they  discover. 

The  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
(ARI)  has  been  tasked  to  fill  this  need.  They  are  to  develop  a  methodolo 
for  conducting  TPEs,  and  guides  for  revising  training  to  eliminate  flaws 
programs.  The  methodology  Is  to  be  applicable  generally  in  training 
programs  for  a  variety  of  weapon  systems.  As  a  vehicle  for  their  develop¬ 
ment,  however,  ARI  selected  the  training  program  for  the  XM1  tank  system. 
The  XM1  was  well  into  the  operational  testing  (OT)  cycle,  and  the  work  to 
be  done  could  be  accommodated  to  the  OT  phasing. 

THE  PRESENT  PROJECT 

ARI  contracted  with  Harless  Performance  Guild,  Inc.  (HPGI)  to  develop 
a  prototype  TPE  methodology. *  Though  the  focus  was  on  the  XM1  tank,  the 
methodology  is  generally  applicable  to  a  number  of  Army  weapon  systems  and 
training  programs.  The  effort  did  not  include  the  development  of  proce¬ 
dures  for  correcting  training  deficiencies  discovered  during  TPEs.  That 
task  was  the  province  of  the  project  described  in  this  report.  The  work 
was  to  be  completed  in  three  phases  as  identified  in  the  ARI  Statement  of 
Work: 

Phase  I:  Review  information  documenting  the  XM1  Transition  Training 

Program  (XM1  TTP),  and  conduct  a  training  effectiveness 
evaluation  of  this  program. 

^Harless  Performance  Guild,  Inc. 

(Working  Paper  FKFU 

for  the  Behavioral  and  Social  Sciences,  November  1979. 


Program  Evaluation 
Research  Institute 


Guidelines  for  Conducting  a  Training 
80-1) .  Fort  Knox,  KY:  U.S.  Army 


Phase  II:  Derive  training  implications  from  the  Phase  I  effectivenes 
evaluation  and  develop  guides  documenting  the  process  used 
to  derive  the  implications. 

Phase  III:  Identify  revisions  to  the  XM1  TTP  from  analyses  of  the 
Phase  II  implications  and  develop  guides  documenting  the 
process  used  to  identify  the  revisions. 

Work  on  Phase  I  was  documented  in  a  separate  report. *  As  explained  in 
that  report,  it  was  not  possible  to  conduct  a  TPE  of  the  XM1  TTP  because  the 
operational  testing  had  already  been  conducted  and  the  documentation 
describing  its  conduct  was  incomplete.  Specifically,  it  could  not  be 
determined  what  tasks  had  been  practiced  during  training,  the  nature  of 
practice  activities,  the  performance  standards  met  during  training,  or  how 
the  program  was  implemented.  Therefore,  Phases  II  and  III  of  the  project 
could  not  build  specifically  upon  documented,  empirically  established  rela¬ 
tions  of  training  practices  and  TPE  outcomes.  As  a  consequence.  Phase  II 
was  revised  and  Phase  III  was  eliminated. 

During  the  revised  Phase  II,  guidance  was  to  be  developed  for 
correcting  the  instructional  deficiencies  that  result  in  performance  defi¬ 
ciencies  on  a  final  test  following  training.  In  keeping  with  the  goal  of 
general  applicability  of  the  methods  involved,  the  trailing  deficiencies 
were  to  be  addressed  not  as  problems  in  XM1  training  per  se,  but  as  short¬ 
comings  that  could  arise  in  any  training  program  where  skill  performance  is 
the  concern.  The  guidance  would  be  such  that  it  could  be  followed  by 
persons  who  are  not  expert  in  training  technology. 


Seville  Research  Corporation.  Phase  I  -  Training  Effectiveness 
Evaluation  of  the  XM1  Operational  Test  II  Transition  Training  Program 
(Letter  Report!.  Pensacola,  FL:  Seville  Research  Corporation,  October 
1979. 


APPROACH 


fe  D 

B  «- 


To  provide  an  Integrated  view  of  sources  of  performance  deficiencies, 
it  was  necessary  to  consider  training  programs  as  a  whole.  In  addition  to 
problems  arising  from  instructional  practices  per  se,  inadequacies  of 
training  content  were  addressed,  as  were  common  difficulties  arising  from 
lack  of  administrative  support  and  absence  of  facilitative  policies. 

The  entire  training  and  TPE  efforts  were  viewed  as  a  system,  and  a  variety 
of  factors  that  can  result  in  less  than  adequate  skill  performance  were 
examined  as  they  impact  the  total  training  system.  Furthermore,  a  scheme 
was  developed  to  guide  training  analysts  in  pinpointing  shortcomings  in 
instructional  practices  so  that  corrections  can  be  specific  to  the 
problems. 

Training  As  A  System 

A  "system"  is  an  integrated  enterprise  in  pursuit  of  a  goal.  It  is 
integrated  in  that  no  part  of  the  system  can  be  viewed  meaningfully  by 
itself.  Each  aspect  depends  on  the  proper  functioning  of  all  other  aspects. 
As  a  system,  a  training  program  has  the  goal  of  preparing  trainees  for 
quality,  efficient  job  performance.  Whether  it  achieves  this  goal  depends, 
first,  on  the  nature  of  training  objectives.  They  must  reflect  job  perfor¬ 
mance  requirements  in  such  a  way  that  trainees  who  achieve  the  objectives 
perform  on  the  job  as  desired.  Next,  the  design  of  the  training  should 
maximize  the  likelihood  that  trainees  will  achieve  the  objectives.  In 
doing  so,  training  practices  and  conditions  must  be  adapted  to  the  charac¬ 
teristics  of  trainees,  their  aptitudes,  entry  skill  levels,  learning 
styles,  and  attitudes. 
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In  fulfilling  these  conditions,  provisions  must  be  made  for  adequate, 
timely  administrative  support.  Personnel  and  other  resources  needed  for 
training  should  be  available.  Policies  and  practices  concerning  the  design 
and  management  of  the  training  program  should  facilitate  every  aspect  of 
the  training  process,  ranging  from  the  selection  and  preparation  of 
instructors  to  a  regular  monitoring  of  training  quality  and  timely  correc¬ 
tive  actions  when  problems  are  discovered. 

An  essential  characteristic  of  a  smoothly  functioning  system  is  an 
effective  communication  network.  If  each  aspect  depends  on  every  other 
aspect,  any  shortcomings  or  changes  in  one  will  have  consequences  for  the 
others.  For  example.  If  training  and  test  contents  differ,  an  analyst 
should  attempt  to  identify  the  reason  for  the  disparity  so  that  something 
can  be  done  either  about  the  test  or  the  training.  He  should  find  out 
whether  the  test  content  reflects  job  requirements  that  are  within  the 
scope  of  the  training  program.  If  so,  training  objectives  should  be  exam¬ 
ined  for  appropriateness  and  revised  as  needed.  The  contents  of  practice 
sessions  must  be  revised  accordingly,  and  training  procedures  adapted  to 
them.  The  entry  characteristics  of  students  must  be  examined  in  terms  of 
the  new  training  objectives,  and  if  necessary,  special  training  must  be 
provided  to  overcome  entry  skill  deficiencies. 

On  the  other  hand,  if  the  test,  not  the  training  content,  is  inappro¬ 
priate,  the  communication  network  should  provide  for  the  analyst  to  notify 
someone  in  authority  so  the  test  can  be  revised.  When  viewed  as  a  system, 
the  test  is  an  integral  part  of  the  training  program  whether  or  not  the 
test  and  the  program  are  under  control  of  the  same  personnel . 

Following  this  orientation,  several  training  deficiencies  are 
discussed  more  than  once  in  the  present  report,  for  often  a  problem  and  its 


correction  can  have  implications  In  more  than  one  area  of  responsibility. 
Even  simple  misuses  of  feedback  during  practice,  for  example,  can  Indicate 
a  need  for  administrative  action  to  Improve  Instructor  selection  or 
training.  Although  a  training  analyst  may  not  be  Involved  In  correcting 
sources  of  some  problems,  he  is  expected  to  distinguish  them  from  Instruc¬ 
tional  deficiencies  per  se.  The  view  of  training  as  a  system  provides  a 
perspective  for  making  the  distinction  through  an  understanding  of  all 
training  deficiencies. 

Identification  of  Instructional  Deficiencies 

Practical  and  coherent  measures  for  correcting  instructional  short¬ 
comings  require  a  method  for  pinpointing  what  goes  wrong  during 
Instruction.  More  than  likely,  corrections  would  involve  insightful  appli¬ 
cations  of  Instructional  technology  to  training  methods.  That  is,  uses  of 
practice,  guidance,  and  feedback  would  have  to  be  adapted  to  exploit  the 
profound  contributions  these  factors  make  to  training.  The  types  of  obser¬ 
vations  required  to  diagnose  instructional  deficiencies  involving  these 
factors  may  not  always  be  possible.  However,  solutions  of  instructional 
problems  must  be  specific  to  the  sources  of  difficulty  if  they  are  to  be 
effective.  Therefore,  personnel  responsible  for  correcting  instructional 
shortcomings  should  attempt  to  make  the  necessary  observations. 

Specifically,  such  personnel  should  observe  trainees  during  training 
practice  so  they  will  be  able  to  tell  when  progress  typically  stops,  what 
is  incorrect  in  the  students'  performance,  and  what  skill  elements  are 
omitted.  If  they  are  familiar  with  the  skills  being  taught  and  know  what 
trainees  have  gone  through  up  to  this  point,  1,e.,  are  familiar  with  the 
earlier  training,  the  personnel  responsible  for  correcting  instructional 
shortcomings  can  likely  spot  the  source  of  the  difficulty  more  easily,  and 
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devise  corrective  measures  accordingly.  The  key  Is  the  continued  progress 
of  trainees,  or  their  lack  of  progress.  At  the  time  progress  stops.  It  Is 
fairly  easy  to  find  out  what  trainees  cannot  do.  By  reflecting  on  the  pre¬ 
ceding  training  activities,  the  source  of  the  problem  can  be  narrowed  down 
quickly. 

The  approach  to  developing  measures  for  correcting  training  deficien¬ 
cies  taken  here  thus  involved  (1)  a  focus  on  the  entire  training  program, 
and  (2)  a  schema  for  identifying  instructional  shortcomings  and  for 
adapting  practice,  guidance,  and  feedback  to  correct  them. 


ORGANIZATION  OF  THE  REPORT 

This  report  has  four  major  sections  of  which  the  present  one  is  the 
first.  Section  II  presents  and  explains  15  principles  regarding  the  use  of 
practice,  guidance,  and  feedback.  A  training  analyst  should  be  thoroughly 
familiar  with  these  principles  when  he  conducts  a  TPE,  determines  sources 
of  performance  deficiencies,  and  identifies  revisions  needed  in  instruc¬ 
tional  practices.  Section  III  examines  instructional  deficiencies  that 
might  be  indicated  by  items  of  evidence  listed  by  HPGI.  Ways  to  pinpoint 
sources  of  training  problems  are  explained  together  with  corrective 
measures  that  focus  on  the  sources.  Section  IV  discusses  five  areas  of 
administrative  responsibility  that  are  critical  to  effective  training 


programs. 


II.  PRINCIPLES  OF  PRACTICE,  GUIDANCE,  AND  FEEDBACK 


All  skill  learning  Is  a  complicated  process.  When  It  Is  necessary  to 
design  a  training  program,  or  to  find  out  why  an  existing  program  Is  not  as 
effective  as  it  should  be,  the  complexity  of  skill  learning  must  usually  be 
considered.  In  a  few  instances,  the  underlying  complexity  may  be  no 
serious  problem.  The  trainees  who  are  to  learn  the  skills  may  already  have 
had  considerable  experience  in  closely  related  skills,  permitting  the  new 
skills  to  be  learned  easily  and  quickly.  This  would  not  make  skills,  or 
their  learning,  less  complex;  rather,  the  experienced  trainees  would 
already  have  gone  through  the  complex  process  of  learning  the  older  skills 
that  make  new  skills  easier  to  master. 

When  performance  deficiencies  occur,  however,  It  means  that  some  part 
of  the  complex  process  of  learning  has  not  been  provided  for  adequately. 
Either  special  attention  must  be  paid  to  the  entry  skill  levels  of  trainees 
and  the  training  adapted  to  them,  or  serious  errors  in  training  practices 
must  be  corrected.  In  either  case  It  is  necessary  to  address  the  process 
of  learning  itself.  Fortunately,  the  complexity  of  learning  can  be  readily 
understood  and  managed  by  thinking  in  terms  of  three  basic  factors  that 
determine  learning:  guidance--the  trainee  must  be  told  or  shown  v»hat  he  is 
to  do;  practice--he  must  do  it;  and  feedback-- he  must  be  able  to  recognize 
when  he  performs  correctly  and  when  he  does  not. 

Obviously,  guidance,  practice,  and  feedback  have  to  be  considered 
together.  Practice,  for  example,  is  useless  unless  the  trainee  is  prac¬ 
ticing  the  right  things.  It  is  through  explaining  and  demonstrating  how 
to  do  a  task,  i.e.,  guiding  him,  that  we  ensure  that  he  is  practicing  the 
right  things.  He  also  needs  to  know  how  he  is  getting  along.  Is  he 


doing  the  task  correctly?  If  not,  what  is  he  doing  wrong?  Feedback, 
properly  used,  answers  these  questions. 

When  training  programs  are  successful,  practice,  guidance,  and  feed¬ 
back  are  being  provided  for  adequately.  When  instructional  deficiencies 
become  apparent,  these  three  factors  are  the  first  things  to  consider  In 
determining  why,  and  In  correcting  the  deficiencies.  Familiarity  with  the 
15  principles  discussed  In  this  section  should  help  training  analysts 
understand  what  should  be  done  In  training,  and  the  kinds  of  Instructional 
shortcomings  that  can  occur.  Each  principle  Is  numbered  for  ease  of 
reference  later. 

PRACTICE 

For  learning  to  occur,  the  learner  has  to  do  something.  When  he  Is 
doing  something  specifically  to  learn  a  skill,  his  activities  are  referred 
to  as  practice.  For  almost  all  motor  skills,  hands-on  practice  Is 
necessary  for  successful  training.  However,  what  a  trainee  thinks  about 
while  he  practices  Is  also  important.  The  principles  that  follow  explain 
Important  points  regarding  practice. 

1.  A  person  learns  to  do  whatever  he  practices  in  the  learning 
situation.  If  he  practices  correct  performance,  he  will  learn  correct 
performance;  If  he  practices  incorrect  performance,  he  will  learn  to  per¬ 
form  incorrectly.  It  Is  Important  therefore  for  trainees  to  perform 
correctly  during  hands-on  practice.  Furthermore,  if  during  a  lecture  or  a 
demonstration  prior  to  hands-on  efforts  a  student  practices  in  his  imagina¬ 
tion  actually  seeing,  hearing,  feeling,  and  doing  what  is  being  talked 
about  or  demonstrated,  he  will  build  a  good  conceptual  basis  for  self¬ 
guidance  as  explained  later.  If  instead  he  practices  listening  or 


9 


criterion  skill  performance.  Generally,  practice  Is  Insufficient  If  It  Is 
discontinued  before  criterion  performance  Is  achieved  at  least  once. 

Beyond  that  point,  however,  there  Is  no  magic  formula  that  says  how  much 
practice  is  enough.  Training  developers  should  keep  in  mind  that  it  is  not 
sufficient  for  a  skill  to  be  performed  correctly  at  one  time  If  It  is  per¬ 
formed  incorrectly  at  other  times.  Such  variability  implies  that  a  trainee 
has  not  fully  learned  the  task.  The  experience  of  instructional  personnel, 
l.e.,  of  persons  who  know  first  hand  vrttat  the  problems  are,  provides  a  basis 
for  decisions  regarding  how  much  practice  there  should  be. 

3.  The  length  and  frequency  of  practice  sessions  should  be  adapted  to 
the  task  being  taught,  the  experience  of  the  learner,  and  the  oppor¬ 
tunities  for  performing  the  task  In  the  future.  One  problem  here  Is  the 
role  of  Interference  during  training.  That  is,  task  elements  and  require¬ 
ments  for  correct  performance  often  become  confused  among  themselves.  In 
skill  learning,  this  Interference  often  increases  during  early  practice  of 
an  unfamiliar  task.  This  Is  especially  true  when  the  task  requires  several 
elements  to  be  coordinated  at  the  same  time,  or  vdten  "branching"  is 
Involved,  l.e.,  the  performer  must  make  a  decision  based  on  cues  available 
as  to  irfilch  of  several  alternative  actions  to  take.  For  a  while  it  may 
seem  that  every  new  thing  a  trainee  learns  becomes  confused  with  things  he 
learned  earlier.  In  such  cases.  It  Is  often  necessary  during  early 
learning  to  have  short  but  frequent  practice  sessions  on  each  training 
task,  shifting  to  quite  different  types  of  tasks  at  the  end  of  a  short 
session.  It  may  also  be  necessary  to  practice  only  parts  of  tasks  at 
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first.  Learners  with  a  good  background  of  experience  In  similar  tasks  are 
not  as  likely  to  confuse  task  elements  and  requirements,  so  they  can 
usually  practice  larger  part- tasks  and  for  a  longer  period  of  time  without 
a  break. 

Future  opportunities  to  perform  a  task  help  determine  how  often  prac¬ 
tice  sessions  should  be.  The  concern  here  Is  retention  of  the  skills 
involved.  If  later  training  activities  for  other  tasks  do  not  in  them¬ 
selves  Involve  the  use  of  a  previously  learned  task,  the  older  task  will 
likely  be  partially  forgotten.  Therefore,  provisions  should  be  made  for 
later  periodic  practice  of  any  task  that  is  not  performed  regularly.  (This 
also  holds  for  proficiency  maintenance  after  assignment  to  an  operational 
unit.)  Again,  the  experience  of  Instructional  personnel  provides  a  basis 
for  deciding  how  frequent  practice  should  be. 

4.  Progress  should  be  made  during  practice.  As  obvious  as  this  point 
seems.  It  is  nonetheless  ignored  in  many  training  situations.  Progress  is 
crucial,  for  If  it  does  not  occur,  the  learner  is  practicing  at  least  some 
Incorrect  performance.  Furthermore,  because  progress  should  occur,  the 
failure  to  make  progress  is  the  best  indication  that  something  in  the 
training  needs  correcting.  This  point  was  discussed  in  Section  I  when  it 
was  emphasized  that  causes  of  training  deficiencies  can  be  identified 
easily  and  accurately  If  the  training  process  is  observed.  Section  III 
will  Illustrate  how  this  may  be  done. 

Here  again,  judgments  as  to  whether  progress  Is  occurring  must  be  made 
by  Instructional  personnel,  and  their  experience  with  similar  kinds  of 
training  problems  provides  a  basis  for  making  the  judgments.  A  learner  may 
actually  be  making  progress  even  though  it  is  not  apparent  at  the  moment, 
either  to  him  or  to  an  Instructor.  The  difficulty  may  be  only  a  temporary 


Interference.  One  way  to  find  out  Is  to  have  the  trainee  take  a  five- 
minute  break  when  progress  ceases  during  practice,  and  then  try  again.  If 
progress  resumes,  fine.  If  not.  It  Is  time  to  find  out  what  the  trouble 
Is.  What  specifically  Is  wrong?  Are  the  trainees  missing  cues?  Which 
ones?  Are  they  Interpreting  cues  incorrectly?  In  what  way?  Do  they  know 
what  responses  to  make?  Have  they  learned  to  make  them?  Answers  to  these 
questions,  which  can  only  be  obtained  through  judgments  of  experienced 
instructional  personnel,  reveal  what  might  have  been  missed  earlier  In 
training  and  what  kinds  of  changes  in  teaching  practices  might  be  needed. 

It  may  also  be  found— and  this  is  especially  true  when  complex  skills 
are  first  being  Integrated — that  trainees  have  adequate  skill  knowledge  for 
each  task  element.  It  just  takes  time  to  put  everything  together.  In  such 
cases,  progress  may  not  be  apparent  immediately;  but  a  rapid  burst  of 
progress  will  follow  further  practice  when  skill  elements  fall  into  place. 

5.  The  learner  should  be  intensively  Involved  during  practice. 
Trainees  should  be  led  to  approach--actually  attack— their  tasks  in  full 
earnestness,  for  if  they  practice  lackadaisical  performance  they  will  learn 
lackadaisical  performance.  The  number  of  students  being  served,  and  the 
practice  schedules,  should  not  be  such  that  there  is  Idle  time  during  prac¬ 
tice  sessions,  for  then  socialization  and  daydreaming  compete  with  learning 
the  task  at  hand.  Furthermore,  what  the  trainees  are  practicing  should  be 
meaningful  to  them.  To  maintain  motivation  and  to  provide  a  sense  of 
realism  while  learning,  they  should  see  the  connection  between  what  they  do 
during  practice  and  the  requirements  for  job  performance.  They  should  make 
progress  toward  meeting  those  requirements  during  each  practice  session, 
and  they  should  realize  that  progress  is  occurring.  In  addition,  the 


environment  for  training  should  reflect  an  atmosphere  of  professionalism. 
The  learner  is  there  to  acquire  important  skills,  and  an  instructor's  pri¬ 
mary  responsibility  Is  to  help  him  do  so.  By  each  accepting  and  doing  his 
part  well,  the  learner  grows  toward  a  personal  professional  goal  that  Is 
valuable  to  the  Army;  and  the  Instructor  fulfills  a  personal  professional 
goal  of  being  a  successful  teacher  while  doing  a  job  of  critical  Importance 
for  military  readiness.  (Section  IV  pursues  this  topic  further.) 

Involvement  of  the  learner  during  practice  depends  on  all  these  con¬ 
ditions,  Including  an  instructor  who  takes  personal  pride  in  quality 
teaching. 

GUIDANCE 

Guidance  means  directing  a  learner's  response.  The  response  may  be  a 
muscular  action  such  as  driving  a  tank,  a  perceptual  action  such  as 
scanning  a  terrain  for  a  target,  or  a  mental  action  such  as  interpreting  a 
cue  or  making  a  decision.  Guidance  Is  involved  whenever  a  learner's  atten¬ 
tion  Is  directed  to  what  movements  to  make,  what  to  look  for,  or  how  to 
Interpret  a  cue  or  make  a  decision.  Guidance  Is  also  Involved  when  a 
learner  Is  told  what  not  to  do. 

The  purpose  of  guidance  during  training  is  to  focus  a  learner's  atten¬ 
tion  on  what  to  do  and  how  to  do  It,  and  to  help  him  avoid  inappropriate 
and  Incorrect  actions.  If  he  practices  only  correct  actions,  he  can  learn 
only  correct  actions;  and  If  he  avoids  incorrect  actions,  he  will  not  learn 
to  make  them.  This  point  cannot  be  taken  to  the  extreme.  Some  incorrect 
actions  will  occur,  and  often  they  can  contribute  to  effective  training. 
Skill  mastery  requires  not  only  knowing  what  to  do,  but  also  what  not  to 
do.  There  are  times  when  experience  doing  the  wrong  thing  teaches  the 


learner  not  to  do  It.  It  is  crucial,  however,  that  the  student  know  imme¬ 
diately  when  an  action  Is  Incorrect  (see  the  following  discussion  of 
feedback) . 

The  principles  that  follow  explain  important  points  regarding  the  use 
of  guidance. 

6.  Guidance  is  needed  when  the  learner  cannot  determine  a  correct 
action  on  his  own.  Generally,  this  means  that  guidance  is  most  valuable 
early  In  learning  when  the  student  is  most  likely  to  miss  pertinent  cues 
and  make  incorrect  or  Inadequate  responses.  For  the  same  reason,  guidance 
is  useful  later  in  training  when  practice  begins  on  a  new  task.  It  is  also 
useful  at  any  other  time  when  it  is  apparent  that  the  student  is  unable  to 
progress  on  his  own. 

7.  Guidance  should  direct  the  learner's  attention  to  individual  cues 
and  specific  performance  requirements.  It  is  not  sufficient  just  to 
describe  or  demonstrate  a  skill;  the  learner  should  be  directed  to  note 
those  specific  things  that  make  the  difference  between  correct  and 
incorrect  skill  performance. 

8.  Verbal  knowledge  from  classroom  Instruction,  text  materials, 
memory  aids,  and  job  aids  such  as  equipment  manuals  should  be  presented, 
and  their  use  practiced,  in  ways  that  capitalize  on  their  guiding  value. 
Classroom  instruction  should  focus  on  verbal  knowledge  the  trainees  can 
understand  and  put  to  use  during  hands-on  practice.  Presentations  of 
topics  and  aspects  of  job  performance  that  they  cannot  understand  at  the 
time  should  be  delayed  until  trainees  have  acquired  the  experience  needed 
to  understand  them.  (Audio-visual  aids  can  help  provide  meaning,  as  can 
previous  experience  with  training  or  operational  equipment.)  Training 
materials  should  be  designed  not  just  to  lead  a  student  through  a  task,  but 
to  focus  his  attention  on  critical  aspects  of  skill  performance. 


9.  Guidance  should  not  be  used  when  It  is  not  needed.  If  a  learner 
receives  too  much  guidance,  he  learns  to  depend  on  guidance  instead  of  on 
himself.  Therefore,  guidance  In  the  forms  of  lectures  and  demonstrations 
should  be  given  routinely  only  at  those  points  during  training  when  all 
students  are  likely  to  need  It.  Additional  guidance  should  be  on  an  indi¬ 
vidual  basis,  and  then  primarily  when  a  learner  stops  making  progress,  or 
for  final  "polishing"  of  a  skill. 

10.  Self-guidance  should  be  a  specific  goal  in  all  training.  A  self- 
guided  performer  Is  what  Is  meant  by  an  Independent  performer.  It  means 
that  he  can  direct  his  own  actions,  recognize  and  correct  his  own  error 
tendencies,  and  that  he  Is  competent  in  the  skill  generally.  The 
self-guided.  Independent  performer  has  to  understand  what  he  is  doing. 

While  we  teach  for  this  goal,  we  seldom  spell  It  out  in  this  manner.  We 
are  more  likely  to  achieve  truly  Independent,  self-guided  skill  performance 
If  we  do  define  it  as  a  goal  so  that  it  is  always  in  mind.  Training  for 
understanding  can  then  be  designed  for  a  clear  purpose. 

FEEDBACK 

As  necessary  as  practice  Is  for  learning.  Its  value  depends  entirely 
on  what  happens  while  practice  occurs.  One  learns  a  skill  not  by  just 
trying  to  perform  It,  but  by  finding  out  which  attempts  are  correct  and 
which  ones  are  not.  In  other  words,  the  learner  must  receive  information- 
feedback— that  lets  him  know  what  Is  correct  in  his  actions  and  what  is 
not.  Otherwise,  there  Is  no  basis  for  continuing  correct  actions  and 
dropping  out  the  Incorrect  ones.  The  learner  would  not  know  the 
difference. 

Depending  on  the  newness  of  the  task  to  be  learned,  and  on  the  pre¬ 
vious  experience  of  the  learner,  a  trainee  may  be  able  to  recognize  when  a 
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skin  has  been  performed  correctly.  As  a  rule,  however,  there  will  be  many 
aspects  of  the  skill  that  he  cannot  judge  at  first.  For  this  reason,  an 
objective  of  every  successful  training  program,  whether  recognized  or  not. 
Is  that  the  trainees  learn  to  provide  their  own  feedback,  l.e.,  to 
recognize  the  difference  in  what  happens  naturally  when  a  skill  is  per¬ 
formed  correctly  and  when  It  Is  not.  Furthermore,  they  must  be  able  to 
recognize  the  feedback  while  performance  is  going  on  so  that  timely 
adjustments  In  performance  can  be  made. 

While  acquiring  this  ability,  the  trainee  will  need  a  lot  of  help.  An 
Instructor  may  have  to  tell  him  what  is  right  and  wrong,  and  point  out  what 
the  trainee  should  notice,  l.e.,  guide  him,  so  that  he  can  learn  the  feed¬ 
back  himself.  In  many  training  situations,  and  especially  during  hands-on 
practice,  such  Individual  attention  will  require  small  trainee/ instructor 
ratios.  Many  modern  training  devices  can  substitute  for  an  instructor  in 
this  role,  and  in  some  cases  do  a  better  job  because  they  can  monitor  what 
trainees  do  more  closely  and  accurately  than  an  instructor  can. 

Accurate  and  timely  feedback  is  so  crucial  to  successful  training  that 
deficiencies  in  student  performance  following  training  almost  always  result 
when  feedback  is  inadequate.  One  must  watch  progress  during  training,  and 
identify  what  goes  wrong  and  when,  to  determine  whether  feedback  is  misused 
and  how  It  should  be  adapted.  The  problem  often  reduces  to  instructors' 
having  too  many  trainees  to  work  with  individually  {see  Section  III),  but 
probably  the  more  common  difficulty  is  that  instructors  do  not  know  how 
Important  feedback  Is  nor  how  to  provide  it. 

The  principles  below  summarize  some  of  the  most  Important  aspects  of 
feedback  in  training. 

11.  Feedback  should  be  specific  to  the  response.  If  feedback  is  to 
let  a  learner  know  whether  he  Is  right  or  wrong,  he  must  be  able  to  tell 
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which  response  Is  right  or  wrong.  If  all  responses  are  correct,  of  course, 
telling  a  student  that  he  performed  the  entire  task  correctly  also  tells 
him  that  each  response  was  correct.  But  if  errors  occur,  a  general 
statement  that  he  "did  It  wrong"  does  not  tell  him  what  parts  of  his  actions 
must  be  corrected.  He  needs  to  know  what  was  right  and  what  was  wrong. 

12.  Feedback  must  occur  while  the  learner  still  remembers  clearly 
what  he  did.  In  practicing  new  tasks  especially,  trainees  often  are 
groping  and  do  not  realize  exactly  what  they  are  doing.  Responses  may  be 
simply  of  the  wrong  sort.  They  may  be  too  slow  or  too  fast,  too  weak  or 
too  strong,  too  early  or  too  late--or  even  omitted.  Except  for  omissions, 
the  trainee  may  have  no  clear  memory  a  few  seconds  later  of  just  how  he 
responded;  and  if  not,  he  would  need  feedback  without  any  delay.  To  the 
extent  he  can  remember  how  he  responded,  delays  in  feedback  are  of  no  great 
consequence. 

13.  When  the  ongoing  coordination  of  an  action  depends  on  continuous 
feedback  from  each  part  of  the  action,  no  delay  in  feedback  (other  than 
that  characteristic  of  operational  performance)  can  be  tolerated.  In  such 
cases  the  feedback,  often  through  the  "feel"  of  an  action,  is  a  cue  for  the 
next  immediate  response.  Necessarily,  the  next  response  cannot  be  ini¬ 
tiated  until  its  cue  occurs.  (This  principle  is  of  primary  concern  when 
simulators  of  actual  equipment  are  used  for  training  motor  tasks.  A  need 
for  realistic  natural  feedback  may  require  a  degree  of  equipment  fidelity 
that  is  not  necessary  for  tasks  where  visual-motor  coordinations  are  not 
crucial .) 

14.  Feedback  should  be  provided  in  ways  that  help  the  learner  see 
when  he  is  making  progress.  In  many  cases  the  trainees  will  be  able  to 
recognize  progress  because  they  can  see  for  themselves  when  they  are 
improving.  This  is  especially  true  when  feedback  that  occurs  naturally  as 


a  result  of  actions  can  be  correctly  interpreted  by  them.  It  is  also  true 
for  simple  sequential  procedural  tasks  when  trainees  have  memorized  the 
steps  or  when  a  checklist  is  available,  for  then  they  can  check  what  they 
do  against  their  memory  or  the  list.  But  in  many  other  cases,  novice 
trainees  especially  cannot  recognize  when  they  are  improving,  even  In  pro¬ 
cedural  tasks  if  they  involve  interpreting  subtle  feedback  cues.  In  these 
instances,  trainees  must  be  told  when  they  are  responding  correctly,  either 
by  an  instructor  or  through  automated  monitoring  by  a  training  device.  And 
at  some  points,  they  will  need  instructor  recognition  of  their  progress  to 
maintain  motivation.  If  the  instructor  cares,  if  the  progress  is  valued  by 
others,  trainees  are  more  likely  to  care  themselves  and  remain  intensively 
involved  in  the  learning. 

15.  When  errors  are  made,  or  when  there  is  little  or  no  progress  in 
skill  acquisition,  the  feedback  should  focus  on  correcting  the  difficulties, 
not  punishing  the  learner.  This  is  one  respect  in  which  the  professionalism 
of  instructional  personnel  as  mentioned  earlier  is  crucial.  If  they  see 
their  responsibilities  as  helping  trainees  and  supporting  their  learning 
efforts,  they  will  use  feedback  only  to  this  end.  That  is,  they  will 
correct  mistakes  in  ways  that  the  trainees  can  recognize  the  instructors' 
concern  for  their  welfare.  Punishing,  humiliating  remarks  by  instructors 
are  never  interpreted  by  trainees  as  helpful.  Reprimands  may  be  appropriate 
when  trainees  are  goofing  off,  but  they  can  never  help,  only  discourage, 
when  they  are  trying. 

SUMMARY 

Fifteen  principles  regarding  the  use  of  practice,  guidance,  and  feed¬ 
back  were  presented  and  explained.  These  principles  will  be  referred  to 
often  in  the  next  section  because  they  underlie  both  the  identification  and 
the  correction  of  shortcomings  in  instructional  practices. 
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III.  IDENTIFYING  AND  CORRECTING  INSTRUCTIONAL  DEFICIENCIES 


This  section  examines  common  shortcomings  in  skill  training  that  arise 
during  instruction.  The  HPGI  report  listed  more  than  20  such  training 
deficiencies.  Each  of  these  deficiencies  is  covered  in  one  or  more  of  20 
instructional  shortcomings  discussed  below.  While  the  discussions  apply  to 
individual  trainees  having  difficulties,  problems  will  be  of  concern  for 
present  purposes  only  if  the  deficiencies  characterize  a  large  enough 
number  of  trainees  to  require  revisions  in  training  practices,  not  just 
special  help  for  individuals. 

In  line  with  a  point  emphasized  in  Section  I,  and  again  when  practice 
was  discussed  in  Section  II,  failure  to  progress  during  practice  is  the 
most  common  focus  for  identifying  instructional  deficiencies.  For  the 
first  14  deficiencies,  steps  for  corrective  actions  are  outlined  in  terms 
of  amount  of  practice,  the  practice  schedule,  and/or  what  is  being 
practiced.  Guidance  and  feedback  are  discussed  as  appropriate  in  connec¬ 
tion  with  the  provisions  for  practice.  The  dual  role  of  practice  as  a 
determiner  of  learning  in  and  of  itself,  and  as  a  "vehicle"  for  productive 
uses  of  guidance  and  feedback,  should  thus  be  evident. 

Issues  of  practice  are  not  directly  Involved  in  the  last  six  defi¬ 
ciencies.  They  are  concerned  with  guidance  and/or  feedback  independently 
of  the  practice  formats.  Their  identification  and  corrections  will  be 
discussed  accordingly. 

The  discussion  of  the  first  instructional  deficiency  immediately 
below  describes  a  general  scheme  for  identifying  sources  of  deficiencies. 
Five  "symptoms"  of  training  problems  are  identified,  each  of  which  is 


discussed  as  it  applies  specifically  to  the  absence  of  sufficient  hands-on 
practice.  Also,  the  implications  of  various  symptoms  for  inadequate  uses 
of  guidance  and  feedback  are  indicated.  To  avoid  unnecessary  repetition, 
later  discussions  will  frequently  refer  back  to  this  scheme  for  the  iden¬ 
tification  of  symptoms,  and  in  some  instances  for  their  interpretation. 
Additional  indicators  of  instructional  deficiencies  and  their  interpreta¬ 
tion  will  be  introduced  as  they  apply  to  each  of  the  other  sources  of 
training  deficiencies. 

1.  There  is  an  Absence,  or  an  Insufficient  Amount,  of  Hands-on 
Practice  During  Training 

This  deficiency  becomes  obvious  when  (1)  no  hands-on  practice  is 
provided  at  all;  (2)  trainees  do  not  know  what  to  do  in  the  presence  of 
suitable  cues;  (3)  their  performance  is  unduly  variable  from  one  trial  to 
another;  (4)  they  are  unduly  slow  in  performing  a  task;  or  (5)  they  depend 
too  much  on  job  aids  and  memory  "joggers."1 

The  presence  or  absence  of  each  of  these  indicators  helps  determine 
what  is  needed.  If  trainees  receive  no  practice  at  all  it  obviously  should 
be  provided.  If  they  receive  some  practice  but  still  do  not  know  what  to 
do,  guided  practice  is  needed  (Principle  6).  If  performance  during  prac¬ 
tice  or  on  a  test  is  unduly  variable,  there  has  not  been  enough  independent 


*An  instructional  deficiency  also  exists  when,  after  being  away  from  a 
task  for  a  while,  trainees  cannot  remember  how  to  do  it  although  they  per¬ 
formed  the  task  adequately  during  earlier  practice.  This  deficiency  may 
not  relate  to  the  formal  training  program  per  se,  but  it  does  indicate  that 
additional  training  is  needed.  If  previously  learned  skills  have  been 
forgotten,  they  obviously  need  periodic  practice  (Principle  3).  Such  prac¬ 
tice  can  be  arranged  at  intervals,  with  Intervals  not  so  long  that 
forgetting  from  one  practice  occasion  to  the  next  results  in  serious  skill 
deterioration.  It  may  also  be  feasible  to  find  ways  to  incorporate  the 
skill  into  the  practice  of  other  skills,  but  the  other  approach  would 
generally  be  less  disruptive  of  an  established  training  routine. 
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practice,  and/or  the  trainees  have  not  learned  to  Interpret  natural  feed¬ 
back  so  as  to  correct  erroneous  response  tendencies  before  they  disrupt 
smooth,  consistent  performance.  Provide  additional  practice  and  note 
whether  progress  Is  made  (Principle  4);  if  not,  provide  guidance  as  needed 
to  help  trainees  Identify  appropriate  feedback  (Principle  7)  and  then 
continue  practice  without  guidance  until  correct  performance  stabilizes. 

Unduly  slow  performance  during  practice  or  on  a  test  can  be  due  simply 
to  not  enough  practice,  or  to  an  excessive  dependence  on  unnecessary 
"crutches"  (e.g.,  checklists  used  as  training  aids).  If  the  former,  pro¬ 
vide  additional  practice;  If  observation  reveals  the  latter  to  be  the  case, 
withdraw  the  "crutches"  (Principles  9  and  10).  The  last  statement  also 
applies  to  the  fifth  indicator  above  concerning  too  much  dependence  on 
performance  aids. 

The  steps  involved  In  correcting  these  deficiencies  thus  depend  on 
their  source.  The  general  scheme  is  to  (1)  provide  practice  if  none  Is 
given,  and  additional  hands-on  practice  if  that  given  Is  not  sufficient; 

(2)  note  whether  trainees  progress  steadily  to  a  desired  level  of 
performance;  (3)  if  not,  provide  minimum  guidance  for  responses  and  for 
interpreting  natural  feedback  cues;  (4)  withdraw  guidance  and  continue 
practice,  using  additional  guidance  sparingly  and  only  when  the  trainees 
cannot  progress  on  their  own;  and  (5)  revise  training  procedures  to  conform 
to  the  changes  so  as  to  achieve  accurate,  stable  performance. 

2.  There  is  Inadequate  Integration  of  Academic  and  Hands-on  Training 

This  deficiency  often  Is  due  to  relying  on  lectures  when  hands-on 
practice  would  be  better.  Insofar  as  lack  of  sufficient  hands-on  practice 
is  the  problem,  this  deficiency  is  essentially  the  same  as  the  one 
discussed  immediately  above,  and  the  same  corrective  steps  are  involved. 
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A  second  source  of  this  deficiency  is  the  frequent  separation  of  content 
for  academic  and  hands-on  training.  That  Is,  one  kind  of  subject  matter  Is 
covered  during  lectures,  and  a  different  kind  during  skill  practice.  The 
understanding  trainees  acquire  should  directly  support,  and  be  usable  In, 
actual  skill  performance.  Verbal  knowledge  and  hands-on  skills  should  form 
a  coordinated,  mutually  supporting  base  for  skill  achievement.  (See 
discussions  of  Principles  8  and  10.)  Training  therefore  should  focus  on 
both,  slighting  neither,  and  with  the  goal  of  maximum  utilization  of 
understanding  while  performing  tasks. 

A  problem  In  the  integration  of  academic  and  hands-on  training  can  be 
identified  in  two  ways.  First,  an  examination  of  the  content  of  academic 
instruction  will  reveal  to  an  experienced  training  analyst  or  subject 
matter  expert  whether  It  is  relevant  to  skill  performance.  Second, 
observing  trainees  during  practice  as  just  discussed  can  reveal  lack  of 
adequate  understanding  of  skill  requirements.  (See  especially  diagnostic 
step  1  regarding  trainees'  not  knowing  what  to  do  in  the  presence  of 
suitable  cues.)  If  inadequate  academic  content  is  the  problem,  an  analysis 
should  be  made  of  the  knowledge  needed  to  perform  properly— relevant  cues, 
contingencies  for  actions,  responses  to  be  made,  standards  for  performance— 
and  lectures  and  demonstrations  revised  accordingly  (Principle  8).  Given 
adequate  academic  content,  then  the  diagnostic-corrective  steps  summarized 
at  the  end  of  the  discussion  of  the  preceding  topic  should  be  followed. 

3.  Session  Times  Are  Inadequate  for  Objectives 

This  deficiency  may  occur  because  too  much  is  attempted  in  the 
training  program,  or  because  of  inefficient  training  methods  and  conditions 
If  the  problem  is  that  too  much  is  attempted,  the  solution  is  obvious.  A 
systematic  analysis  is  needed  to  establish  reasonable  training  goals  and 
priorities  (see  Section  IV). 
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If  Inefficient  training  methods  are  the  culprit.  It  means  that 
progress  during  training  Is  slower  than  It  could  be.  All  training  should 
be  monitored,  and  tasks  and  stages  of  training  where  trainee  progress  Is 
unduly  slow  should  be  noted.  Diagnostic  studies  as  described  earlier 
should  be  made  If  needed,  and  corrective  actions  should  be  taken  as 
Indicated.  There  are  so  many  possible  inefficient  training  methods  that  It 
is  not  practical  to  pursue  them  further  here.  However,  all  other  training 
deficiencies  discussed  in  this  section  involve  Inefficient  training  prac¬ 
tices  which  In  many  cases  tend  to  extend  the  time  required  for  training. 

The  reader  is  referred  to  the  other  discussions  for  things  to  look  for 
while  observing  trainee  progress,  how  to  Interpret  what  may  be  found,  and 
how  to  correct  instructional  shortcomings. 

4.  Practice  Activities  Lack  Realism 

Realism  can  be  lacking  for  a  number  of  reasons.  At  one  extreme,  tasks 
that  depend  on  physical  actions  and  sequencing  and  coordinating  reponses 
may  be  taught  largely  through  verbal  description.  If  so,  the  discussions 
under  Topics  1  and  2  above  are  relevant:  Hands-on  practice  should  be  pro¬ 
vided  as  needed  to  achieve  stable  correct  performance,  and  the  verbal 
knowledge  should  be  Integrated  with  the  physical  actions  Involved  in 
performance. 

Realism  is  a  broader  Issue  than  this,  however.  First,  efficiency  in 
training  typically  requires  Instruction  under  conditions  that  do  not  hold 
during  testing.  For  example,  demonstrations  and  artificial  feedback  are 
used  during  training;  performance  during  practice  can  be  interrupted  to 
focus  instruction  on  particular  points;  mistakes  can  be  easily  "forgiven"; 
etc.  During  a  test,  however,  the  student  is  on  his  own,  performance  must 
progress  realistically,  and  mistakes  are  counted  against  the  performer. 


Furthermore,  just  the  fact  that  a  test  Is  a  test  makes  It  different  from 
most  training  activities  In  the  perception  of  trainees. 

A  training  program  should  recognize  these  differences  and  prepare 
trainees  not  only  to  perform  skills,  but  to  perform  them  under  conditions 
that  will  hold  during  testing.  Generally,  a  "dry  run"  Is  called  for  prior 
to  the  test  Itself  so  that  skills  can  be  Interfaced  with  test  conditions. 

In  the  absence  of  a  dry  run  or  a  comparable  means  for  adapting 
learning  to  test  conditions,  the  training  analyst  can  have  difficulty 
ascertaining  the  source  of  a  performance  deficiency  on  a  test,  and  hence 
prescribing  a  correction.  It  makes  a  difference  whether  the  foundation 
training  was  adequate  and  test  failures  were  due  to  lack  of  opportunities 
to  adapt  to  the  test  situation,  or  the  foundation  training  Itself  was 
faulty.  If  the  former,  the  correction  requires  only  a  dry  run  or 
equivalent;  If  the  latter,  diagnoses  of  shortcomings  in  all  aspects  of  the 
Instruction  may  be  needed  (see  first  topic  above). 

A  second  problem  concerning  realism  is  becoming  more  and  more  preva¬ 
lent  In  military  training.  Specifically,  equipment  designed  to  exploit 
principles  of  training  technology,  and  to  reduce  training  costs,  often  dif 
fers  from  operational  equipment  In  physical  and  functional  characteristics 
If  trainees  practice  with  such  equipment  but  are  tested  on  actual 
equipment,  the  Interfacing  of  skills  learned  with  testing  conditions  be¬ 
comes  critical.  More  than  a  single  dry  run  may  be  needed  before  trainees 
can  demonstrate  what  they  actually  learned.  Thus,  Identifying  the  source 
of  a  test  performance  deficiency  is  more  involved,  and  it  usually 
requires  expertise  in  training  evaluation.  Such  analyses  are  beyond  the 
scope  of  the  present  effort,  but  TPEs  in  the  near  future  may  well  have  to 
expand  to  Include  them. 
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5.  There  Is  an  Absence,  or  an  Insufficient  Amount,  of  Integrated 
'  Practice 

A  learner  may  have  mastered  every  part- task  that,  taken  together,  make 
up  a  whole  task,  but  still  not  be  able  to  perform  well  on  the  whole  task. 

The  problem  concerns  what  Is  being  practiced,  and  how  much.  Principle  11s 
clearly  relevant:  A  trainee  cannot  learn  to  Integrate  parts  of  a  task 
unless  he  practices  the  Integration  Itself.  Therefore,  Integrated  task 
performance  itself  must  be  practiced.  Furthermore,  integrative  practice 
must  continue  until  the  Integration  is  stable  and  can  be  retained 
(Principles  2  and  3) . 

To  Identify  this  training  deficiency,  determine  (1)  whether  trainees 
practice  the  whole.  Integrated  task;  (2)  if  so,  whether  there  Is  steady 
progress  to  a  desired  proficiency  level;  (3)  whether  whole-task  skills  once 
achieved  are  retained;  and  (4)  if  original  whole-task  skills  are  not 
retained,  identify  which  part-tasks  (If  any)  are  forgotten. 

If  too  little  practice  is  the  only  problem,  then  progress  will  be 
steady  during  the  practice  that  is  provided  for  the  whole  task.  If  steady 
progress  does  not  occur,  refer  to  earlier  discussions  of  insufficient 
hands-on  practice  for  ways  to  identify  inadequate  uses  of  guidance  and 
feedback.  The  present  problem  is  exactly  the  same  except  that  integration 
of  performance  is  the  "task"  to  be  learned.* 

(If  the  separate  skills  of  a  complex  task  have  been  mastered,  their 
integration  into  a  complete  task,  once  mastered  also,  will  usually  be 
retained  fairly  well.  Integration  imposes  an  organization  on  the  task,  and 
organization  is  a  primary  basis  for  retention.  Sequential  tasks  such  as  those 

*The  fourth  step  immediately  above,  while  implicit  in  the  earlier 
discussion,  is  explicit  here  for  reasons  apparent  in  subsequent  discussions. 
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Involved  In  many  procedures  may  not  acquire  a  high  degree  of  organization 
even  when  they  appear  to  be  fairly  well  learned.  To  aid  retention.  It  Is 
necessary  that  parts  of  a  sequence  depend  on  each  other  In  their  execution. 
The  fact  that  one  step  follows  another  In  a  procedural  task  does  not 
necessarily  make  the  steps  depend  on  each  other  In  trainees1  understanding. 
Therefore,  procedural  skills  may  be  forgotten  If  not  practiced  periodically.) 

How  much  practice  In  Integration  Is  needed  for  learning  to  take  place  Is 
revealed  by  trainees'  progress  during  Integrative  practice.  After  profi¬ 
ciency  Is  first  achieved,  additional  periodic  practice  may  well  be  needed 
for  retention  (Principle  3).  The  experience  of  instructional  personnel 
with  training  the  tasks  Involved  can  often  provide  bases  for  judgments 
regarding  the  amount  of  additional  practice  needed.  But  two  points  should 
be  remembered.  First,  If  a  skill  that  was  well  integrated  earlier  In 
training  has  deteriorated  to  the  extent  that  recovery  Is  slow,  the  problem 
Is  probably  due  to  the  deterioration  of  skill  components-- various  part- 
tasks--not  to  a  substantial  loss  in  the  integration  Itself.  Second,  during 
training,  sequential  skills  may  not  achieve  the  true  Integration  required 
for  retention,  so  periodic  practice  may  be  necessary  to  assure  correct 
performance  on  a  final  test. 

The  learning  of  task  Integration  requires  less  total  practice  If 
Integrated  practice  begins  early,  I.e.,  if  separate  part- tasks  are  not 
practiced  longer  than  necessary.  As  soon  as  progress  in  integration  can  be 
made  (Principle  4),  larger  and  larger  groupings  of  part-tasks  should  be 
practiced  together  In  the  manner  In  which  they  must  be  performed  on  the 
job.  It  will  also  aid  later  integration  If,  during  earlier  practice  of 
each  part- task,  trainees  are  made  constantly  aware  of  the  total  context  In 
which  the  part- task  must  be  performed  (see  discussion  of  Principle  5). 
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They  should  be  reminded  as  often  as  necessary  to  think  about  overall  task 
requi rements  while  practicing  a  part- task. 

The  steps  for  correcting  a  deficiency  Involving  the  absence,  or  an 
insufficient  amount,  of  Integrative  practice  are,  then:  (1)  provide 
Integrative  practice,  and  do  so  as  soon  as  part-tasks  have  been  learned 
well  enough  for  steady  progress  In  Integration  to  be  possible;  (2)  monitor 
integrative  practice  to  ensure  that  progress  Is  maintained  up  to  an  accept¬ 
able  level;  (3)  if  progress  Is  disrupted,  Identify  the  troublesome  part- 
tasks  involved  and  provide  additional  part-task  practice  as  necessary;  (4) 
determine  for  the  whole  task  and  for  part-tasks  whether  additional  periodic 
practice  Is  needed  after  original  proficiency  Is  achieved  so  as  to  assure 
retention  of  the  complete  skill;  and  (5)  revise  the  training  schedule 
accordingly. 

6.  Step  Sizes  Are  Too  Large 

Step  size  refers  to  the  number  of  task  elements  that  are  presented  at 
one  time,  or  to  the  complexity  of  cues  and  responses  that  must  be 
recognized  and  performed  precisely  in  a  step.  Difficulties  arise  when 
trainees  cannot  deal  with  all  the  task  elements,  cues,  and  responses 
simultaneously.  In  other  words,  they  are  required  to  do  more  than  they  are 
prepared  to  tackle  at  the  time. 

Step  sizes  that  are  too  large  can  introduce  a  number  of  problems.  When 
they  are  being  demonstrated,  trainees  can  easily  confuse  task  elements,  and 
forget  cues  and  required  actions  before  they  have  a  chance  to  practice  them 
(see  discussions  of  Principles  7  and  8).  If  too  many  new  things  must  be 
done  at  one  time  or  completed  quickly  as  a  sequence,  it  may  be  impossible 
to  provide  feedback  for  specific  acts,  or  as  soon  as  it  is  needed 
(Principles  11  and  12). 


The  identification  of  step  sizes  that  are  too  large  Is  straight¬ 
forward:  (1)  observe  the  points  during  hands-on  practice  where  trainees 

have  undue  difficulty  just  getting  started;  (2)  determine  whether  prior 

✓ 

guidance,  oral ,  written,  or  by  demonstration,  was  provided;  and  (3) 
question  the  trainees  to  determine  what  confused  them  and  when,  or  whether 
they  understand  any  of  it. 

While  the  result  of  (1)  Is  all  that  is  needed  to  determine  what  step 
sizes  are  too  large,  (2)  and  (3)  help  determine  what  to  do  about  the 
problem.  It  Is  possible,  of  course,  that  the  last  two  diagnostic  actions 
will  reveal  deficiencies  in  guidance  and  in  timely,  specific  uses  of  feed¬ 
back  that,  when  corrected,  will  remove  the  difficulty.  In  other  words,  step 
sizes  may  not  be  too  large  if  care  is  taken  to  be  sure  trainees  get  the 
help  needed  for  each  element. 

If  it  appears  that  improved  guidance  and  feedback  cannot  solve  the 
problem,  then  step  sizes  should  be  adjusted.  Either  task  complexity  at 
given  points  should  be  temporarily  reduced,  or  smaller  segments  of  the 
sequence  of  task  elements  should  be  practiced  separately.  In  either  case, 
the  result  is  to  focus  practice  on  part- tasks.  It  is  important,  however, 
that  the  separate  part-tasks  be  properly  chosen.  First,  any  part-task 
grouping  must  make  sense  so  that  the  separate  elements  can  be  organized 
meaningfully.  The  integrity  of  the  part- task  and  the  task  as  a  whole  must 
not  be  compromised.  Second,  as  discussed  under  Principle  3,  the  problem  is 
largely  confusion  of  cues  and  response  requirements  (interference),  so 
part-tasks  should  be  such  that  confusion  is  minimized  during  early 
training.  Third,  part-tasks  should  be  small  enough  for  trainees  to 
remember  the  details  of  any  guidance  they  receive.  Fourth,  task  elements 
included  in  a  given  step  should  be  easily  distinguishable,  by  the  instruc¬ 
tor  and  by  the  trainee,  for  purposes  of  specific,  timely  feedback 
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(Principles  11  and  12).  Fifth,  the  past  experience  of  the  trainees  should 
be  considered  in  organizing  part- tasks  for  practice  (Principle  3).  When 
possible,  each  part- task  should  contain  elements  for  which  the  trainees 
already  have  some  proficiency,  plus  elements  that  are  new  to  them.  The 
ability  to  perform  portions  of  a  part-task  well  will  help  In  learning  the 

unfamiliar  portions.  Finally,  of  course,  the  entire  task  should  be 

practiced  as  a  whole  (see  earlier  discussion  of  task  integration). 

The  steps  for  correcting  this  deficiency  are,  then:  (1)  identify 
groupings  of  task  elements  that  do  not  compromise  the  integrity  of  the 
task,  and  that  capitalize  on  trainees'  past  knowledge;  (2)  redesign  the 
demonstrations  to  provide  specific  guidance  as  discussed;  (3)  redesign 
feedback  to  focus  on  separate  task  elements;  (4)  observe  progress  during 
practice  to  ensure  that  desirable  learning  is  occurring;  (5)  to  check  ade¬ 
quacy  of  the  new  task  steps,  determine  if  trainees,  after  a  reasonable 
amount  of  practice,  can  perform  each  step  unaided  (Principle  10);  and  (6) 
provide  for  recombining  part-tasks  for  practice  in  larger  units,  or  as  a 

whole  task,  as  soon  as  progress  can  be  made  on  the  larger  units. 

7.  Complexity  or  Novelty  of  Tasks  is  not  Considered  from  Trainees' 
Viewpoint 

"Complex"  tasks  are  those  requiring  fine  visual  discriminations,  pre¬ 
cise  motor  skills,  high  rates  of  performance,  and/or  long  series  of  steps 
to  be  recalled.  "Novel"  tasks  are  those  for  which  trainees  have  no 
relevant  prior  experience. 

A  point  to  bear  in  mind  regarding  either  task  complexity  or  novelty  is 
that,  from  the  standpoint  of  task  performance,  no  task  is  complex  to  an 
expert  in  the  task  or  novel  to  a  person  who  has  relevant  prior  experience. 
Conversely,  all  tasks  are  complex  to  the  extent  a  person  has  not  learned  the 
skills  involved,  and  are  novel  when  encountered  for  the  first  time.  The 
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reason  for  these  statements  Is  to  emphasize  that  the  same  principles  of 
learning  are  involved  regardless  of  task  complexity  or  task  uniqueness,  and 
that  trainees'  previous  experience  should  be  a  major  factor  in  devising 
training  techniques.  In  a  few  cases,  it  may  be  clear  from  past  experience 
in  teaching  a  skill  or  others  similar  to  It  that  the  approach  used  in  a 
given  training  program  was  inadequate.  In  most  cases,  however,  better 
diagnostic  information  will  be  needed  and  it  can  be  obtained  by  observing 
trainees'  progress  as  has  been  emphasized  in  foregoing  discussions.  The 
analytic  steps  decribed  for  deficiency  1  are  applicable  to  the  present 
problem.  Similarly,  the  use  of  part-task  training  as  described  immediately 
above  for  adjusting  step  sizes,  and  the  integration  of  part-tasks  as 
discussed  earlier,  are  clearly  relevant  as  approaches  to  training  complex 
and  novel  tasks.  As  pointed  out  in  all  these  discussions,  the  skill  ele¬ 
ments  and  stages  of  training  that  are  troublesome  must  be  identified,  and 
disruptions  in  performance--or  failure  to  make  any  progress--is  the  clue. 

The  corrective  steps  to  be  taken  would  be  determined  by  what  was 
observed  during  the  monitoring.  In  most  cases,  part-task  training  will 
be  needed  to  break  the  complexity  down  into  manageable  units  (see  earlier 
discussion  of  step  sizes).  If  the  complexity  derives  from  fine  discrimi¬ 
nations  among  competing  cues  and  types  of  natural  feedback,  or  from  dif¬ 
ficult  coordinations  of  actions,  special  part- task  training  (or  training 
devices)  should  focus  on  them.  Novel  tasks  also  may  need  to  be  broken  into 
small  steps.  Guidance  may  need  to  be  more  detailed  and  given  more  often 
than  for  other  tasks,  and  the  instructor  may  have  to  provide  feedback  him¬ 
self  while  the  trainees  are  learning  to  recognize  and  interpret  natural 
feedback. 

8.  Training  is  Geared  Too  High  for  Trainees'  Entry  Knowledge  and  Skill 

If  trainees  entering  a  training  program  do  not  have  the  background  it 


was  assumed  they  would  have  when  the  program  was  designed,  they  will  have 
undue  difficulties  during  training.  This  is  primarily  an  administrative 
problem  in  that  a  redesign  of  the  earlier  stages  of  training  is  needed  (see 
Section  IV).  In  the  meantime,  however,  trainees  will  need  special 
attention.  The  point  to  bear  in  mind  is  that  no  new  principles  of  training 
are  involved.  The  15  principles  given  in  Section  II  apply  to  all  levels  of 
previous  achievement  and  ability.  The  special  attention  referred  to 
involves  adapting  the  principles  to  take  care  of  unanticipated  trainee 
weaknesses. 

The  corrective  cteps  depend  to  some  extent  on  the  reasons  for  low 
entry  skills.  There  are  two  general  reasons:  a  lack  of  requisite  prior 
experience,  and  low  aptitude  for  the  skills  to  be  learned.  Two  general 
types  of  diagnostic  efforts  are  needed.  First,  experienced  training  per¬ 
sonnel  should  review  trainees'  records  to  pinpoint  as  much  as  possible  the 
nature  of  their  backgrounds  and  general  level  of  aptitude.  Formal  and 
informal  discussions  with  entering  trainees  would  help  clarify  possible 
gaps  in  requisite  experiences.  Also,  formal  or  informal  proficiency  tests 
covering  required  beginning  skills  would  reveal  specific  deficiencies. 

The  second  type  of  diagnostic  effort  would  concentrate  on  how  the 
trainees  actually  progress  during  training.  The  general  scheme  for  obser¬ 
ving  practice  discussed  for  deficiency  1  should  be  followed.  Interpre¬ 
tations  of  the  observations  would  focus  specifically  on  indications 
of  inadequate  experience  versus  low  aptitude.  Inadequate  experience  is 
indicated  when  trainees  can  progress  normally  after  remedial  training  to 
provide  the  missing  skills.  Low  aptitude  is  indicated  when  progress  is 
unduly  slow  regardless  of  remedial  training.  (It  would  also  be  slow  during 
remedial  training.) 
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To  the  extent  the  problem  Is  due  to  inadequate  requisite  experience, 
corrections  would  typically  involve  providing  the  needed  experiences  before 
training  begins,  or  later  in  training  just  so  long  as  the  remedial  training 
is  given  before  the  requisite  skills  involved  are  to  be  used.  In  some 
cases,  special  training  techniques  may  remove  the  need  for  remedial 
training,  i.e.,  the  deficiencies  can  be  overcome  while  the  regular  tasks 
are  being  taught.  For  example,  trainees  may  not  know  how  to  use  par¬ 
ticular  tools  required  to  perform  a  task;  but  they  may  be  able  to  learn  to 
use  the  tools  while  practicing  the  task,  so  remedial  practice  with  the 
tools  alone  would  not  be  necessary.  However,  alterations  in  usual  teaching 
procedures  would  likely  be  required.  The  types  of  changes  involved  would 
be  generally  the  same  as  the  adaptations  needed  for  trainees  of  low 
aptitude,  the  topic  of  the  next  paragraph. 

To  trainees  of  low  aptitude,  and  to  those  who  participate  in  regular 
training  even  though  they  do  not  have  certain  requisite  prior  experiences, 
the  skills  to  be  taught  will  appear  quite  complex,  even  novel.  Therefore, 
the  earlier  discussion  of  these  topics  is  relevent  here.  Step  sizes 
are  likely  to  be  critical,  and  steps  that  are  too  large  should  be  a  special 
concern  while  monitoring  progress  during  practice.  It  is  also  likely  that 
demonstrations,  and  overviews  presented  at  the  beginning  of  training 
sessions,  were  designed  with  higher  entry  level  skills  in  mind.  If  so, 
they  will  probably  need  immediate  revision.  Instructors  should  assume  less 
ability  to  understand  and  simplify  their  presentations  accordingly.  It  is 
also  necessary  that  customary  ways  of  providing  feedback  be  reviewed  care¬ 
fully.  Trainees  with  low  entry  skills  will  surely  have  difficulty  pro¬ 
fiting  from  feedback  that  is  not  highly  specific  and  timely  (Principles  11 
and  12).  They  will  also  need  more  guidance  in  learning  to  recognize  and 


interpret  feedback  that  occurs  naturally  during  task  performance 
(Principle  7) . 

Finally,  these  trainees  are  likely  to  be  sensitive  to  the  interference 
that  builds  up  during  early  stages  of  learning,  so  practice  schedules  should 
be  carefully  adapted  to  ensure  steady  progress  (Principles  4  and  14).  And, 
of  course,  practice  should  be  closely  monitored  so  that  difficulties  can  be 
identified  without  delay.  Many  of  these  trainees,  especially  those  of  low 
aptitude,  will  be  easily  discouraged  when  they  are  making  no  headway.  It 
is  critical  for  them  to  stay  intensively  involved  (Principle  5). 

The  steps  for  alleviating  this  training  deficiency,  then,  are:  (1) 
determine  the  reason (s)  for  low  entry  skills;  (2)  provide  preliminary  reme¬ 
dial  training  as  needed;  (3)  follow  procedures  for  reducing  step  sizes  and 
complexity  and  novelty  of  tasks  discussed  earlier;  (4)  revise  uses  of 
guidance  and  feedback  as  needed  to  make  them  specific,  timely,  and 
meaningful;  (5)  adapt  practice  schedules  and  monitor  progress  closely  to 
identify  and  correct  difficulties  as  soon  as  practical  after  they  arise; 
and  (6)  initiate  a  revision  of  the  training  program  so  that  tentative 
corrective  steps  can  be  thoroughly  integrated  into  a  systematic  program 
(see  Section  IV). 

9.  Job  Aids  Needed  But  Are  Not  Used  in  Training 

If  aids  are  to  be  used  on  the  job,  and  students  are  expected  to  know 
how  to  use  them  upon  completing  training,  students  should  practice  using 
them  while  training.  This  deficiency  results  from  simply  ignoring 
Principle  1:  Trainees  have  to  practice  a  skill  to  learn  it.  If  the  use  of 
job  aids  is  not  practiced,  their  use  cannot  be  learned.  Instructors  often 
slight  job  aids  even  when  they  are  included  in  training,  perhaps  preferring 
to  use  limited  time  on  less  mechanical  aspects  of  performance.  They  may 
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feel  that  job  aids  can  be  learned  on  the  job.  Also,  some  Instructors 
appear  to  believe  that  true  learning  requires  that  everything  must  first 
"be  In  the  head,"  that  job  aids  are  all  right  as  reminders  but  should  not 
substitute  for  knowledge.  The  result  Is  that  students  often  have  to 
memorize  by  rote  the  Information  contained  In  job  a1ds--and  rote  memory  Is 
notoriously  unreliable. 

It  is  easy  to  determine  if  a  given  job  aid  is  not  used  at  all: 

Observe  the  instruction,  ask  the  Instructors  If  they  use  them,  and/or 
determine  if  the  trainees  even  know  what  they  are.  If  job  aids  are 
included  in  training,  the  adequacy  with  which  trainees  learn  to  use  them  is 
easily  assessed  by  observing  them  during  practice:  Do  they  know  when  to 
use  them?  Do  they  use  them  correctly?  Are  they  properly  Integrated  with 
the  tasks  to  which  they  relate?  Assuming  trainees  learn  to  use  them  satis¬ 
factorily  while  training,  can  they  use  them  later  during  the  test?  (This 
last  question  concerns  relatively  short-term  retention.  While  it  could  be 
asked  regarding  any  instructional  deficiency  discussed,  it  can  be  espe¬ 
cially  important  where  job  aids  are  concerned.  Job  aids  often  involve  rote 
learning  of  procedures  rather  than  full  understanding  of  the  factors 
involved.  Rote  learning  is  very  unstable  without  periodic  practice.  Even 
if  the  test  is  delayed  only  a  few  hours  or  a  day,  significant  forgetting 
can  occur,  and  additional  practice  may  be  needed.) 

Some  general  comments  regarding  practice  and  guidance  are  also  in 
order.  First,  as  with  any  other  skill,  guidance  is  usually  needed  in 
learning  to  use  a  job  aid,  so  Principles  6,  7,  and  8  are  clearly 
applicable.  Many  job  aids  include  Instructions  for  their  use,  in  which 
case  it  is  necessary  only  to  assure  that  trainees  can  follow  the 
instructions.  (Some  additional  guidance  may  be  needed,  however.) 
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Second,  trainees  should  eventually  demonstrate  ability  to  use  the  aids 
without  help  from  other  sources  (Principle  10).  Third,  periodic  practice 
using  the  training  aids  will  probably  be  needed  If  trainees  use  them  mecha¬ 
nically  with  little  understanding  of  the  relation  of  the  aids  to  equipment 
functioning  and  task  performance. 

The  correction  of  tills  training  deficiency  is  simple:  (1)  determine 
which  aids  are  to  be  used  during  the  final  test,  l.e.,  are  to  be  mastered 
during  training;  and  (2)  for  those  aids  whose  use  is  included  In  the  test, 
provide  sufficient  practice  during  training  to  assure  the  proficiency 
desired. 

10.  Trainee/Equipment  Ratio  Is  Too  Large 

The  concern  here  is  that  the  training  equipment  available  may  not  be 
sufficient  in  amount  for  trainees  to  get  the  hands-on  practice  needed. 

This  is  essentially  a  matter  of  insufficient  hands-on  practice  which  was 
discussed  earlier  (deficiency  1),  and  the  scheme  presented  there  for 
determining  whether  enough  practice  Is  provided  applies  here.  The  iden¬ 
tification  of  the  problem  is  more  straightforward,  however.  If  progress 
continues  so  long  as  trainees  practice,  but  trainees  do  not  reach  criterion 
achievement  levels  or  remember  what  they  learn,  then  there  has  not  been 
enough  practice.  And  at  the  same  time,  if  training  equipment  is  being  uti¬ 
lized  fully,  there  is  not  enough  training  equipment,  at  least  for  the 
training  program  as  designed.  (In  the  extreme  case,  some  trainees  may  not 
have  the  opportunity  for  hands-on  practice  at  all.  In  such  a  case,  it  is 
obvious  that  more  equipment  is  needed.) 

Obtaining  a  desirable  trainee/equipment  ratio  is  primarily  an  admin¬ 
istrative  matter.  More  equipment  can  be  procured,  class  sizes  reduced,  or 
training  time  increased.  Nevertheless,  more  efficient  use  of  available 
equipment  often  can  relieve  some  training  problems. 


A  common  barrier  to  solving,  or  at  least  alleviating,  this  problem  Is 
an  Insistence  on  a  lock-step  training  regimen:  All  students  In  a  class  are 
expected  to  learn  the  same  things  at  the  same  time.  The  program  Is  planned 
as  a  series  of  training  events,  and  the  sequence  of  skills  practiced  has  to 
be  the  same  for  all  trainees.  Therefore,  there  Is  a  high  demand  for  every 
type  of  equipment  when  times  come  for  everyone  to  use  each  type.  If  there 
must  be  a  standard  sequence,  and  if  equipment  is  in  short  supply,  students 
might  enter  the  training  at  staggered  times  rather  than  as  complete 
classes. 

However,  the  training  could  be  planned  so  that  different  trainees 
could  be  doing  different  things  at  any  one  time.  It  would  be  necessary  to 
ensure  that  prerequisite  skills  are  learned  in  sequence,  but  even  with  this 
restriction  it  is  usually  possible  to  have  alternative  tracks  within  a 
single  training  program.  (There  may  be  additional  demands  on  instructional 
personnel,  however.)  Part-tasks  as  discussed  earlier  offer  a  number  of 
opportunities  for  alternative  simultaneous  training  activities.  Also,  some 
relief  from  demands  on  training  equipment  can  often  be  obtained  by  using 
proficiency  advancement,  i.e.,  letting  a  trainee  stop  a  given  type  of  prac¬ 
tice  when  he  first  shows  adequate  achievement,  beginning  him  on  a  new  task, 
and  turning  the  equipment  over  to  another  trainee.  Proficiency  advancement 
requires  monitoring  each  trainee,  or  small  groups  of  trainees,  which  can  be 
time  consuming;  but  with  experience  the  extra  responsibility  would 
generally  be  negligible.  Trainees  could  even  monitor  themselves  in  many 
cases,  and  call  on  an  instructor  only  when  they  think  they  can  demonstrate 
skill  proficiency. 

Principles  2,  3,  and  4,  taken  as  a  whole,  can  ensure  quality  training 
in  a  proficiency  based  program  or  one  with  alternative  tracks:  Practice 


must  be  sufficient  for  learning  and  retention,  but  the  amount  can  be 
adapted  to  the  learner's  need;  and  practice  can  occur  according  to  varying 
schedules--durations  and  frequency--so  long  as  adequate  progress  In 
learning  Is  maintained. 

11.  Training  Materials  Are  Poor  or  Inaccurate 

The  most  common  sources  of  this  difficulty  are  dated  (obsolete)  lesson 
plans,  equipment  manuals,  and  job  aids;  dated  or  otherwise  Inappropriate 
training  equipment;  and  otherwise  appropriate  training  equipment  that  Is  In 
such  poor  repair  that  it  does  not  function  as  real  equipment  does,  or  that 
has  been  used  so  much  that  cues  have  been  made  unduly  obvious.  When  these 
conditions  exist.  Instructors  must  either  use  training  materials  as  they 
are  or  devise  their  own  alternatives.  Neither  approach  is  desirable. 

These  problems  are  primarily  administrative  matters,  and  they  are 
discussed  in  Section  IV.  Nevertheless,  some  stopgap  measures  are  possible. 
Determine  which  available  materials  and  equipment  can  be  used  successfully 
and  which  cannot.  Rarely  is  an  entire  manual  dated,  for  example,  unless 
the  equipment  has  been  completely  changed.  And  even  dated  or  poorly 
maintained  training  equipment  may  still  be  usable  for  teaching  some  skills. 
In  any  case,  any  discrepancies  between  training  and  operational  materials 
should  be  clearly  explained  to  the  trainees,  and  some  practice  should 
require  them  to  recognize  which  parts  are  applicable  and  which  are  not. 

If  the  equipment  is  to  provide  feedback,  it  should  be  used  only  for  those 
tasks  where  correct  and  incorrect  actions  can  receive  feedback  accordingly 
(Principle  11),  and  where  the  feedback  can  be  provided  in  a  timely  manner 
(Principles  12  and  13). 

12.  Training  Becomes  Boring 

Boredom  may  be  due  to  the  instructor's  manner  of  teaching  or  to  the 
length  of  practice  sessions.  Also,  some  tasks  are  plain  tedious.  Even 


though  many  tasks  will  remain  boring  on  the  job,  there  are  good  reasons 
for  avoiding  boredom  during  training. 

Boredom  is  indicated  when  trainees  not  only  stop  progressing  during 
training,  but  go  downhill.  They  become  careless,  forgetful,  and  are  easily 
distracted.  If  boredom  is  due  to  the  task,  most  Instructors  will  have 
bored  trainees  when  the  task  is  practiced,  but  not  when  certain  other  tasks 
are  being  taught.  If  the  instructor  is  the  problem  his  trainees  will  tend 
to  be  bored  regardless  of  the  task,  while  trainees  under  other  Instructors 
will  be  bored  less  often.  An  additional  factor  should  also  be  considered. 
It  is  not  uncommon  for  trainees  to  be  required  to  practice  tasks  after  they 
have  achieved  proficiency  on  them.  This  is  likely  to  occur  when  all  of  the 
trainees  are  expected  to  practice  each  task  according  to  a  rigid  schedule. 
Trainees  who  learn  rapidly  or  who  begin  with  high  entry  levels  of  similar 
skills  must  thus  continue  practice  even  though  it  is  not  needed.  These 
trainees  will  likely  go  through  the  motions  with  a  minimum  of  attention  and 
involvement. 

Boredom  Is  detrimental  to  training.  The  trainee  Is  not  properly 
involved  (Principle  5),  and  lackadaisical  performance  is  likely  to  be 
practiced  and  learned  (Principle  1).  It  does  not  matter  that  the  trainee 
will  later  be  bored  on  the  job  too.  The  Idea  is  to  practice,  and  learn,  a 
skill.  Boredom  does  not  have  to  be  learned,  so  no  practice  is  needed  for 
it.  Training  can  come  to  grips  with  tedium  after  trainees  have  acquired 
the  skills  needed  to  do  their  jobs. 

If  boredom  Is  due  to  previous  mastery  of  the  skills,  it  can  be  easily 
avoided  by  permitting  proficiency  advancement  (see  earlier  discussion  of 
trainee/equipment  ratio).  If  the  boredom  is  due  to  the  instructor,  the 
discussion  of  Instructor  qualifications  in  Section  IV  is  pertinent.  But 
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whether  due  to  the  Instructor  or  the  tedious  nature  of  tasks,  boredom  may 
well  be  reduced  by  careful  planning.  Several  types  of  adjustments  might  be 
made.  (1)  Practice  can  be  arranged  so  that  boring  tasks  are  pursued  during 
a  given  practice  session  only  as  long  as  needed  to  ensure  regular  progress 
from  one  practice  session  to  another.  (2)  Tedious  and  Interesting  tasks 
can  be  practiced  alternately.  (3)  An  atmosphere  of  operational  realism  can 
be  Introduced  Into  the  session.  (4)  Elements  of  challenge  can  be  Introduced 
through  competition  among  trainees,  teams  of  trainees,  or  competition  of 
each  trainee  with  himself  to  surpass  earl ler  performance.  (Competition  Is 
a  more  effective  motivator  when  every  trainee  has  a  reasonable  chance  to 
"win,"  and  when  there  is  a  formal  recognition,  tangible  or  otherwise,  of 
superior  performance.) 

These  approaches  focus  on  ways  to  avoid  boredom  by  beneficial  arrange¬ 
ments  of  practice  and  practice  conditions.  Even  with  Ideal  practice  con¬ 
ditions  and  schedules,  however,  some  tasks  will  remain  boring.  Maintaining 
motivation  can  overcome  the  boredom  that  cannot  be  avoided. 

Students  are  likely  to  stay  motivated  and  involved  during  practice 
If  they  recognize  that  they  are  making  progress  toward  a  valued  goal 
(Principle  5).  If  they  cannot  recognize  this  progress  themselves, 
Instructors  or  automated  feedback  systems  in  training  devices  should  help 
the  students  recognize  when  they  are  progressing  (Principle  14).  In  addi¬ 
tion,  the  instructor,  not  a  device,  should  encourage  the  student  by 
recognizing  his  progress  (see  discussion  of  Principle  14);  and  when  correc¬ 
tions  are  needed,  the  feedback  should  be  given  in  a  supporting,  not 
punishing,  manner  (Principle  15).  By  the  example  of  their  instructors,  and 
with  instructor  understanding  and  encouragement,  trainees  can  learn  to 
accept  tedium  and  adjust  to  it  constructively.  A  general  atmosphere  of 
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professionalism  (see  discussion  of  Principle  5)  Is  conducive  to  this 
acceptance  and  adjustment. 

13.  Training  Schedule  Is  Interrupted 

For  a  variety  of  reasons--equipment  breakdowns,  temporary  unavailabil¬ 
ity  of  instructors,  delays  due  to  *eather--interruptions  in  training  may 
well  lower  final  achievement  unless  their  effects  are  corrected. 

Whether  a  training  schedule  has  been  interrupted  is  easy  enough  to 
determine.  The  effects  of  the  Interruption  are  the  Issue.  Overall 
training  plans  should  provide  guidance  for  dealing  with  likely  Inter¬ 
ruptions,  and  bases  for  such  guidance  are  the  concern  of  this  discussion. 

The  first  task  is  to  anticipate  the  kinds  of  difficulties  that  might 
result  from  an  Interruption.  Many  problems  can  be  overcome  by  an  oral 
review  of  the  training  completed  prior  to  an  interruption,  accompanied 
perhaps  by  demonstrations  of  key  skills.  It  may  also  be  desirable  to 
design  a  practice  session  in  which  the  students  actually  perform  the 
earlier  skills.  If  serious  disruptions  in  performance  occur,  additional 
practice  on  troublesome  skills  can  then  be  provided.  If  there  are  good 
reasons  (based  on  experience,  for  example)  to  expect  no  serious  disruptions 
in  performance,  the  oral  review,  possibly  with  demonstrations,  may  be 
adequate,  in  which  case  training  can  pick  up  where  it  left  off.  But 
regardless  of  what  corrective  steps  might  be  used  at  first,  subsequent 
practice  should  be  monitored  by  the  instructor  to  assure  progress  as 
discussed  under  deficiency  1. 

The  steps  are:  (1)  review  what  was  covered  during  the  training  prior 
to  the  interruption,  using  demonstrations  when  needed;  (2)  provide 
diagnostic  practice  if  it  is  anticipated  that  appreciable  forgetting  has 
occurred,  and  follow  up  with  additional  practice  on  troublesome  skills  if 
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and  as  needed;  (3)  continue  training  from  the  point  of  interruption;  (4) 
monitor  the  subsequent  training  to  see  whether  additional  review  and 
corrective  practice  are  needed;  and  (5)  provide  additional  corrective 
instruction  as  Indicated. 

14.  Trainees  Forget  Much  of  What  They  Leaned  When  There  is  a  Long 
Interval  Between  Training  and  Testing 

The  clue  to  this  deficiency  is  that  there  was  a  fairly  long  break  be¬ 
tween  the  end  of  practice  on  a  particular  task  and  the  test,  and  that 
trainees  could  not  perform  skills  on  the  test  that  they  could  perform  at 
the  time  they  practiced  It.  An  obvious  remedy  to  this  deficiency  is  to 
reduce  the  interval  so  that  fewer  events  will  interfere  with  retention.  A 
test  administered  immediately  following  practice  will  indicate  whether  the 
task  had  been  learned  at  all,  even  If  It  may  be  soon  forgotten. 

This  problem  probably  should  be  viewed  in  a  broader  perspective, 
however.  If  skills  are  forgotten  during  a  relatively  brief  period  that 
elapses  between  practice  and  being  tested  upon  completing  training,  not 
much  is  likely  to  be  remembered  after  assignment  to  a  field  unit. 

There  is  a  difference,  of  course.  Training  programs  are  designed  to 
provide,  and  quickly,  a  foundation  upon  which  experiences  can  build.  The 
foundation  is  seldom  comprehensive  and  thorough  enough  to  stand  alone. 
Trainees  must  later  encounter  situations  in  which  skills  can  repeatedly  be 
put  to  use  if  the  original  foundation  is  to  serve  its  purpose. 
Opportunities  to  use  some  skills  may  not  arise  for  some  time.  Therefore, 
the  test  following  practice  can  address  only  the  original  foundation,  not 
the  accumulation  of  experiences  necessary  to  build  full  skill  competence. 

Thus,  the  problem  reduces  to  what  the  test  is  to  reveal.  If  it  is  to 
establish  that  the  students  reached  a  given  level  of  proficiency,  however 


"temporary"  It  might  be.  It  should  be  given  Immediately  following  training. 
If  the  test  is  unduly  delayed.  It  simply  makes  the  test  results  less  useful 
for  this  purpose.  After  all,  delaying  a  test  cannot  of  itself  render 
adequate  training  Inadequate. 

If  delayed  tests  are  to  be  used  to  assess  training  achievement,  pro¬ 
vide  periodic  practice  during  the  delay  for  those  skills  that  tend  to  lose 
precision  (Principle  3).  Then  poor  test  results  can  be  interpreted  as  an 
actual  instructional  deficiency. 

15.  Overviews  Preceding  Practice  Are  Inadequate 

A  preliminary  overview  of  a  training  session  can  help  trainees  by 
directing  their  attention  to  the  actions  to  be  learned,  and  to  the  cues  and 
natural  feedback  accompanying  the  actions.  If  an  overview  does  not 
accomplish  this  purpose,  efficiency  of  training  suffers. 

An  overview  is  a  guiding  process.  If  it  is  not  adequate,  trainees 
will  not  know  what  they  are  to  do  (or  understand— overviews  are  needed  for 
lectures).  They  will  miss  cues  and  natural  feedback,  and  not  be  clear  at 
first  on  what  they  mean  even  when  they  are  pointed  out.  Inadequate  over¬ 
views,  therefore,  are  indicated  by  unnecessary  floundering  during  practice, 
and  by  the  need  for  an  instructor  to  provide  unusual  amounts  of  elementary 
guidance  during  practice.  A  common  occurrence  is  for  an  instructor  to 
interrupt  practice  for  an  entire  class  to  explain  basic  points  that  are 
being  missed.  It  may  be,  of  course,  that  trainees  cannot  understand  the 
guidance  until  they  struggle  with  a  task  for  a  while.  Nevertheless,  the 
training  analyst  should  ask  whether  a  more  adequate  overview  would  have 
made  the  Interruption  unnecessary. 

Improving  inadequate  overviews  involves  applications  of  Principles  6, 
7,  and  8.  Overviews  should  be  specific  in  what  the  trainee  is  to  look  for. 
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and  how  he  Is  to  respond  or  otherwise  make  use  of  the  information  when  the 
time  comes.  An  overview  making  use  of  an  outline  of  key  cues  and  actions 
on  a  guide  sheet,  or  written  on  a  chalkboard  while  they  are  being 
explained,  can  be  especially  helpful.  If  trainees  have  trouble 
understanding  what  is  being  said,  demonstrations  of  the  tasks  being 
described  should  be  included  when  feasible. 

16.  Appropriate  Preliminary  Experiences  Are  Not  Provided 

"Preliminary  experiences"  refers  to  those  provided  at  the  beginning  of 

a  lecture  or  practice  session  that  are  needed  to  understand  overviews  and 
task  requirements. 

Symptoms  of  this  problem  are  similar  to  those  resulting  from  poor 
overviews  because  trainees  may  flounder  or  need  additional  elementary 
guidance  during  practice.  However,  this  difficulty  is  indicated  when  a 
trainee  can  recall  points  from  the  overview  that  are  unquestionably 
specific,  but  cannot  use  the  Information.  (The  earlier  discussion  regarding 
entry  skills  identified  additional  indications  of  this  problem.)  Its 
correction  calls  for  identifying  specifically  what  trainees  fail  to 
understand,  either  through  questioning  or  observations  as  discussed 
earlier,  and  arranging  through  demonstrations  or  special  practice  for  the 
meanings  to  be  acquired. 

17.  No  Demonstration  or  Poor  Demonstration  Is  Given 

The  concern  is  that  trainees  have  unnecessary  difficulties  when  they 
are  not  shown  how  a  skill  is  supposed  to  be  performed. 

It  is  easy  to  tell  whether  a  demonstration  is  given.  It  is  not  quite 
so  easy  to  determine  whether  the  demonstration  could  be  seen  and  understood 
by  all  trainees,  whether  it  was  adequate,  or  even  needed.  However,  by 
following  the  diagnostic  scheme  outlined  for  deficiency  1,  the  inadequacies 
of  demonstrations  can  be  identified,  as  well  as  the  need  for  them. 
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Demonstrations  are  needed  when  trainees  flounder  at  the  outset  or 
make  an  undue  number  of  errors.  If  these  things  happen  following  a 
demonstration,  then  the  demonstration  was  not  adequate.  The  nature  of  the 
inadequacies  then  is  the  concern.  It  could  be  that  too  much  was 
demonstrated  at  one  time,  and  the  trainees  either  could  not  remember  it  all 
or  confused  various  task  elements  with  each  other.  The  presentation  of  the 
demonstration  may  have  been  too  rapid,  incomplete,  difficult  to  watch 
because  too  many  trainees  were  around,  or  given  in  such  a  perfunctory 
manner  that  the  trainees  paid  less  than  full  attention  to  it. 

Earlier  discussions  of  skill  complexity,  skill  uniqueness,  step  sizes, 
and  trainees'  entry  skills  identify  what  to  look  for  insofar  as  the  amount 
of  information  covered  in  a  demonstration  is  the  problem.  As  for  prepara¬ 
tion  of  trainees  to  learn  from  the  demonstration,  the  preceding  sections  on 
overviews  and  prerequisite  experiences  are  pertinent. 

Problems  related  to  the  manner  of  presentation  and  the  content  of  the 
demonstration  can  be  identified  most  easily  by  questioning  the  trainees  and 
noting  which  skill  elements  seem  to  be  unnecessarily  troublesome. 

Therefore,  an  attempt  should  be  made  to  interview  trainees  if  problems 
related  to  demonstrations  are  suspected. *  Specifically,  determine  what  the 
trainees  missed,  and  why.  Was  a  missed  skill  element  Included  in  the 
demonstration?  If  so,  did  the  trainees  understand  it  at  the  time  it  was 
demonstrated?  If  they  understood  it  at  the  time,  forgetting  is  the  problem 
and  smaller  steps  are  needed.  If  they  did  not  understand  it,  was  it 
because  it  was  covered  too  rapidly?  Were  some  skill  elements  left  out  or 

^are  should  be  taken  to  assure  that  a  proper  sample  of  trainees  are 
interviewed.  For  example,  trainees  in  some  viewing  positions  and  distances 
may  be  able  to  observe  a  demonstration  while  trainees  at  other  locations 
and  distances  may  not.  This  is  often  a  problem  in  large  classes. 
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glossed  over?  Could  all  trainees  see  the  demonstration  well  enough  to 
follow  it?  Did  they  have  opportunities  to  ask  questions  for  clarification? 

Straightforward  applications  of  Principles  6,  7,  and  8,  as  indicated 
by  answers  to  the  questions,  would  handle  problems  related  to  the  manner  of 
presentation  and  content  of  demonstrations.  If  classes  are  too  large  to 
see  a  demonstration  as  a  single  group,  the  demonstrations  should  be  given 
to  small  subgroups  and  repeated  until  all  trainees  have  been  served.  (If 
multiple  demonstrations  are  necessary,  they  would  have  the  advantage  of 
permitting  trainees  having  unusual  difficulties  to  view  the  demonstration 
more  than  once.) 

18.  Trainees  Do  Not  Have  a  Model  of  Performance  to  Follow 

A  key  clue  to  the  need  for  a  demonstration  of  correct  performance  is 
the  inability  of  trainees  to  tell  by  themselves  when  they  are  doing  a  job 
correctly.  If  they  think  an  incorrect  action  is  correct,  or  if  they  tend 
to  be  quite  variable  in  performance,  they  need  a  model  to  go  by.  This 
problem  can  become  apparent  during  any  stage  of  training.  In  early  stages, 
trainees  may  not  know  what  to  do  at  all.  Later,  they  may  show  evidence  of 
substantial  learning,  even  to  the  point  of  occasional  near-perfect  perfor¬ 
mance.  A  giveaway,  however,  would  be  inconsistent  performance.  They 
would  not  understand  task  requirements  clearly  enough  to  perform  the  task 
correctly  every  time.  They  would  perhaps  miss  finer  points  of  cues  and 
natural  feedback,  and  their  actions  would  lack  the  precision  and  discipline 
required. 

The  preceding  discussion  of  demonstrations  covers  this  problem  in 
early  stages  of  training.  During  later  stages,  model  demonstrations  are 
often  needed  to  accomplish  that  final  "step"  of  honing  and  refining  skills 
(see  discussion  of  Principle  9).  Trainees  must  learn  the  difference  be¬ 
tween  a  correct,  precise  response,  together  with  its  cues  and  timing,  and  an 
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action  that  is  almost,  but  not  quite,  correct.  Highly  precise  feedback  can 
help  make  these  distinctions,  but  illustrations  of  the  differences  using 
ideal  models  of  skill  performance  can  help  trainees  see  what  changes  in 
their  performance  are  required. 

19.  There  Are  No  Criteria  for  Performance  or  the  Criteria  Are  Poor 

If  it  is  not  clear  how  a  skill  should  be  performed,  low  levels  and 
diverse  patterns  of  performance  are  likely.  HPGI  stated  the  problem 
succinctly:  "How  can  we  expect  a  trainee  to  perform  if  he  cannot  detect 
the  difference  between  correct  and  incorrect  performance?"  There  is  no 
basis  for  consistent  guidance  because  there  is  no  way  to  determine  pre¬ 
cisely  what  the  trainee  should  do,  or  how.  Nor  can  feedback  help  trainees 
distinguish  between  correct  and  incorrect  actions  if  the  difference  between 
them  is  not  clear  to  the  instructor. 

Through  repetition  the  preceding  paragraph  belabors  the  issue.  Yet, 
this  deficiency  should  be  emphasized  because  it  is  so  common  in  training 
programs.  More  often  than  not,  training  objectives  do  not  include  a  state¬ 
ment  of  the  behavioral  requirements  that  must  be  fulfilled  for  the  objec¬ 
tives  to  be  met.  And  commonly,  the  objectives  themselves  are  stated  so 
generally  that  they  cannot  guide  training  except  in  a  vague  way  (e.g.,  "The 
trainee  will  engage  a  target  according  to  acceptable  standards").  The 
result  is  that  instructors  develop  intuitive  standards  of  their  own. 
Performance  criteria  thus  vary  from  one  instructor  to  another;  and  because 
the  criteria  are  rarely  spelled  out  in  such  cases,  a  given  instructor  will 
likely  be  inconsistent  himself  in  the  performance  standards  he  sets. 

The  place  to  start  in  identifying  this  deficiency  is  with  formal 
statements  of  training  objectives.  Do  they  specify  what  trainees  are  to  be 
able  to  do  after  training,  the  conditions  and  cues  that  determine  when  they 
are  to  do  it,  and  the  standards  they  must  meet?  If  not,  each  instructor 


must  make  his  own  decisions-- in  which  case,  do  the  instructors  themselves 
spell  out  objectives  in  this  manner?  Are  Instructors  consistent  with  each 
other,  and  with  themselves,  when  they  do  develop  criteria? 

The  training  analyst  should  not  be  satisfied  just  because  he  discovers 
well  defined,  formal  training  objectives  and  performance  criteria.  There 
is  still  the  question  of  whether  instructors  pay  adequate  attention  to 
them.  It  would  generally  be  necessary  to  observe  instructors'  behavior 
during  training  to  clarify  this  issue.  In  some  cases,  questioning  trainees 
regarding  performance  criteria  may  provide  the  needed  information;  but  if 
poorly  prepared  trainees  are  interviewed,  they  may  not  know  specifically 
why  they  are  poorly  prepared.  An  instructor  could  be  employing  criteria 
correctly  when  the  trainees  could  not  tell  whether  he  is  or  not  because 
they  do  not  know  the  difference  between  quality  and  so-so  performance. 

The  correction  of  this  problem  is  primarily  an  administrative  matter. 
The  two  aspects  discussed  here,  poor  training  criteria  and  instructors' 
failure  to  use  criteria,  are  discussed  further  in  Section  IV. 

20.  The  Trainee/Instructor  Ratio  Is  Too  Large 

Training  suffers  when  an  instructor  has  more  students  than  he  can  work 
with  effectively.  This  problem  is  indicated  any  time  individual  trainees' 
difficulties  cannot  be  corrected  because  instructors  do  not  have  the  time 
or  occasion  to  provide  the  individual  attention  needed  for  the  correction. 
Stress  has  been  placed  on  specific  and  timely  uses  of  guidance  and 
feedback.  If  an  instructor  does  not  have  time  to  identify  individual 
trainees'  problems,  and  provide  corrective  instruction,  he  has  too  many 
trainees. 

Of  course,  this  is  not  an  either-or  issue.  In  a  lecture-demonstration 
situation,  the  only  requirements  are  that  all  trainees  be  able  to  hear 


clearly,  see  all  demonstrations  adequately,  and  have  opportunities  to  ask 
questions  for  clarification.  During  hands-on  practice,  on  the  other  hand, 
considerable  individual  attention  is  often  needed  for  effective,  efficient 
training.  This  means  that  more  instructors  would  have  to  be  available  when 
classes  are  large.  However,  beneficial  trade-offs  involving  reduced  class 
sizes  are  also  possible.  Specifically,  students  can  be  trained  in  less 
time  when  individualized  guidance  and  feedback  are  possible.  Proficiency 
advancement  also  becomes  more  feasible,  which  can  save  training  time.  (See 
earlier  discussion  under  trainee/equipment  ratio.)  Fewer  trainees  may  be 
involved  at  a  given  time,  but  what  is  lost  by  reducing  instructor  student 
loads  may  well  be  regained  through  classes  of  shorter  duration,  and  with 
the  probable  extra  benefits  of  more  effective  training  and  longer  retention 

The  correction  of  this  training  problem  must  begin  with  an  administra¬ 
tive  decision  to  determine  the  maximum  number  of  students  an  instructor  can 
handle  effectively,  and  to  reduce  the  student  flow  or  increase  the  number 
of  instructors  accordingly.  It  would  then  be  necessary  to  train  instruc¬ 
tors  to  provide  the  monitoring,  specialized  guidance,  extra  feedback,  etc., 
that  make  for  effective,  efficient  training.  These  topics  are  discussed  in 
Section  IV. 

SUMMARY 

This  section  discussed  the  diagnosis  and  correction  of  20  instruc¬ 
tional  deficiencies.  Schemes  for  diagnosing  instructional  deficiencies 
focused  primarily  on  observations  to  make  during  practice  sessions. 
Interpretations  of  observations  and  corrections  of  deficiencies  were 
according  to  appropriate  adaptions  of  the  15  principles  regarding  practice, 
guidance,  and  feedback  presented  in  Section  II. 
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IV.  TRAINING  DEFICIENCIES  RELATED  TO  ADMINISTRATIVE  PRACTICES 

a 

Section  I  pointed  out  that  a  number  of  problems  that  result  in  poor 
test  performance  after  trainees  complete  training  can  be  traced  back  to 
m  administrative  practices  and  neglect.  In  most  instances,  the  deficiencies 

and  indications  thereof  that  HPGI  classified  as  "not  skill  related"  fall 
into  this  group.  There  are  also  several  deficiencies  covered  in  Section 
III  that  are  primarily  administrative  matters.  However,  the  earlier  treat¬ 
ments  were  concerned  with  reducing  negative  effects  on  training  through 
adjustments  in  teaching  methods,  not  removing  the  sources  of  problems. 

1  This  section  addresses  the  sources  of  these  deficiencies  that  lie  outside 

the  realm  of  instructional  practices  as  such.  That  is,  all  problems  exa¬ 
mined  here  require  administrative  resolve  and  action  for  their  solution. 

E  Whether  or  not  the  correction  of  these  problems  are  a  concern  for  the 

training  analyst,  an  understanding  of  their  nature  and  what  is  involved  in 
their  correction  should  provide  a  useful  perspective  for  him.  Also,  some 
®  discussions  in  this  section  should  help  in  understanding  instructional 

deficiencies. 

The  discussions  are  organized  under  five  heads:  Indications  of  Need  to 
Redesign  Training  Programs;  Training  Support  and  Facilitation;  Instructor 
Characteristics  and  Qualifications;  Management  of  Attitudes;  and  Problems 
in  Performance  Testing. 

INDICATIONS  OF  NEED  TO  REDESIGN  TRAINING  PROGRAMS 

Section  III  discussed  ways  to  revise  and  adapt  instructional  practices 
,  to  overcome  a  number  of  deficiencies  in  trainee  performance.  While  the 

training  adjustments  are  applicable  to  individual  trainees  with  problems, 
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the  emphasis  was  on  problems  shared  by  trainees  in  general.  Thus,  Section 
III  addressed  specific  training  program  revisions  in  considerable  detail. 

Nevertheless,  there  can  be  general  inadequacies  In  program  design  that 
call  for  more  than  specific  revisions.  Rethinking  the  general  design  of  a 
program  may  well  be  required.  Fifteen  indications  listed  by  HPGI  of  a  need 
to  make  substantial  revisions  in  a  training  program  are  listed  below.  They 
vary  in  the  scope  of  Implied  redesign  efforts  required  to  avoid  training 
deficiencies,  but  all  imply  a  need  for  a  systematic  effort  that  will  need 
administrative  action  and  support. 

1.  Pertinent  tasks  are  omitted  from  training. 

2.  There  are  no  performance  criteria,  or  criteria  are  poor. 

3.  Job  aids  should  be,  but  are  not,  included  in  training. 

4.  Trainees  are  tested  on  content  different  from  that  in  the  program. 

5.  Test  performance  criteria  are  higher  than  training  criteria. 

6.  Session  times  are  inadequate  for  material  to  be  covered. 

7.  Trainees  in  the  program  are  different  from  those  for  whom  the 
program  was  designed. 

8.  Trainees'  entry  skill  levels  are  lower  than  those  for  whom  the 
program  was  designed. 

9.  There  is  widespread  failure  on  the  first  test  attempt. 

10.  Trainees  performed  well  on  other  tests  covering  other  skills. 

11.  There  is  a  history  of  poor  performance  by  trainees  completing  the 
program. 

12.  Previous  Training  Effectiveness  Analyses  (now  termed  Training 
Program  Evaluations)  indicated  poor  training. 

13.  Most  persons  familiar  with  the  training  program  rate  it  as  poor  or 
only  fair. 
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14.  An  observer  has  difficulty  describing  the  training  plan. 

15.  An  observer  has  difficulty  correlating  training  tasks  with  the  test. 
The  first  six  items  identify  shortcomings  in  training  objectives  and 


program  implementation.  The  seventh  and  eighth  reveal  that  assumptions 
regarding  trainee  characteristics,  though  perhaps  valid  at  one  time,  no 
longer  hold.  Numbers  9-12  can  indicate  a  general  problem  of  ineffective 
training  that  may  be  due  to  a  number  of  shortcomings.*  The  last  two  imply 
that  there  is  little  rhyme  or  reason  to  what  goes  on  in  training. 

There  are  excellent,  well  documented  procedural  guides  for  developing 
training  programs  that  help  avoid  all  these  problems.  While  it  may  not  be 
necessary  to  follow  all  aspects  of  any  highly  systematized  approach,  a 
general  overview  of  the  process  will  clarify  what  is  involved  and  how  it 
might  be  adapted. 

At  the  outset,  job/ task  analyses  should  identify  what  must  be  known 
and  done  to  perform  a  skill  correctly.  The  conditions  and  cues  calling  for 
the  performance  of  a  task  and  subtasks  comprising  it  are  specified,  and  the 
criteria  that  performance  is  to  meet  are  defined.  The  job/task  analyses 
also  give  an  overall  picture  of  what  is  involved  in  an  operational  job  as 
well  as  details  of  the  specific  tasks  that  must  be  performed.  Hence,  tasks 
can  be  prioritized  for  training  and  a  content  can  be  clearly  defined  for  a 
training  program.  Entry  skills  of  trainees  should  be  assessed  to  avoid 
unnecessary  instruction  on  skills  they  have  already  acquired.  Training 
objectives  can  then  be  derived  to  target  every  skill  and  skill  element 

*The  trouble  could  be  minor,  however,  insofar  as  No.  9  is  concerned. 
See  discussion  in  Section  III  of  need  to  practice  using  skills  before  being 
tested  on  their  use. 

2 

See,  for  example.  Interservice  Procedures  for  Instructional  Systems 
Development  (TRADOC  Pamplet  350-30),  August  1975;  also  Montemerlo  and 
Tennyson,  op.  clt. 


required  for  the  tasks  to  be  trained.  Performance  criteria  would  not  be 
problems  because  they  would  be  defined  during  the  task  analyses. 

A  course  syllabus  would  then  be  defined,  and  training  materials  and 
resources  required  to  Implement  the  syllabus  would  be  Identified.  When 
first  Implemented,  It  would  be  recognized  that  some  changes  would  probably 
be  needed,  so  procedures  for  a  diagnostic  evaluation  of  the  program  would 
be  specified. 

A  program  developed  through  a  systematic  process,  and  implemented 
accordingly,  would  not  have  the  general  shortcomings  indicated  by  the  15 
items  listed  above.  If  tasks  or  job  aids  are  omitted  from  training,  for 
example,  the  omission  would  be  deliberate  and  with  full  understanding  of 
what  would  have  to  be  learned  in  subsequent  training  or  on  the  job.  Thus, 
a  content  is  defined  both  for  training  and  for  performance  testing.  The 
training  program  would  be  geared  to  trainee  characteristics,  including  the 
amount  of  time  scheduled  for  training  each  skill.  Criteria  for  performance 
appropriate  during  training  and  for  testing  would  be  clear.  Trainees  who 
succeed  during  training  could  be  expected  to  succeed  on  the  performance 
tests  because  the  tests  would  key  on  known  training  content  and  performance 
criteria.  And  no  careful  observer  would  have  difficulty  recognizing  a 
coherent  training  plan. 

Depending  on  the  thoroughness  of  the  effort  that  designed  the  program 
originally,  it  may  be  necessary  when  difficulties  arise  to  focus  only  on 
selected  aspects  of  the  program.  If  trainees'  entry  skills  are  much  lower 
than  anticipated,  for  example,  early  remedial  instruction  and  special 
attention  to  early  stages  of  training  (see  Section  III)  may  be  all  that  are 
needed.  However,  if  a  training  program  has  been  in  operation  for  some 
time,  it  is  quite  possible  that  the  skills  being  taught  and  the  training 
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equipment  used  have  become  obsolete  In  some  respects.  (A  formerly  effec¬ 
tive  program  for  which  Items  3,  4,  5,  9,  10,  11,  and  12  become  applicable 
would  suggest  such  an  obsolescence.)  A  more  Inclusive  analysis  and  more 
extensive  program  changes  might  thus  be  needed. 

TRAINING  SUPPORT  AND  FACILITATION 

A  number  of  factors  related  to  inadequate  administrative  support  and 
lack  of  facilitative  policies  can  affect  the  quality  of  training.  Some 
factors  such  as  poor  training  equipment  can  have  direct  negative  impacts  on 
training.  The  effects  of  others  such  as  failure  to  monitor  quality  of 
training  may  be  subtle  in  that  quality  of  training  may  deteriorate  too  gra¬ 
dually  to  be  noticed.  Twelve  such  factors  are  listed  below  that  illustrate 
the  range  of  administrative  responsibility  to  support  and  facilitate 
training.  Each  factor  is  a  fairly  common  problem  in  military  training. 

1.  Trainee/instructor  ratio  too  large. 

2.  Trainee/equipment  ratio  too  large. 

3.  Poor,  obsolete,  or  inaccurate  training  equipment. 

4.  Absence  of  task  documentation. 

5.  Distractive/poor  training  environment. 

6.  Inadequate  maintenance  of  training  equipment. 

7.  Insufficient  numbers  or  quality  of  training  aids. 

8.  Insufficient  numbers  or  quality  of  job  aids. 

9.  Inadequate  space  for  training. 

10.  Failure  to  monitor  quality  of  training  to  assure  effectiveness. 

11.  Poor  communication  among  personnel  at  various  levels  of  respon¬ 
sibility,  and  within  single  levels. 

12.  Failure  to  emphasize  the  importance  of  training  through  visible 
administrative  interest  and  commitment. 
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A  direct  solution  to  these  problems  is  always  desirable  but  not  always 
practical.  The  first  problem  could  be  eliminated,  for  example,  if  the 
nionber  of  instructors  could  be  increased.  A  decrease  in  trainee  flow  would 
also  alleviate  this  problem  as  well  as  the  second  one.  Purchasing 
additional  equipment  would  also  relieve  the  second  problem. 

But  it  is  not  always  possible  to  attack  problems  so  directly,  espe¬ 
cially  when  additional  funds  are  required  or  changes  in  numbers  of  person¬ 
nel  are  entailed.  Nevertheless,  the  point  to  bear  in  mind  is  that 
something  can  be  done  about  every  one  of  these  problems.  All  that  is 
required  is  administrative  resolve  to  seek  a  practical  improvement.  It 
behooves  the  analyst  who  discovers  these  problems  to  call  them  to  the 
attention  of  appropriate  management  personnel  so  that  efforts  toward 
solution  can  be  undertaken. 

Ways  to  alleviate  the  first  three  difficulties  were  discussed  in 
Section  III.  While  the  adjustments  explained  there  were  viewed  as  stopgap 
measures,  thorough  systematic  analyses  of  the  alternatives  presented  may 
well  reveal  improvements  that  are  satisfactory  in  the  long  run.  If  not, 
direct  action  as  mentioned  above  would  be  necessary.  As  for  Item  3,  inad¬ 
equate  training  equipment,  unless  the  inadequacies  are  minor,  more  radical 
adjustments  are  called  for.  If  poor  equipment  results  in  unusable 
training,  or  worse,  training  that  leads  to  erroneous  skill  performance,  it 
would  be  better  to  eliminate  the  equipment  entirely  than  to  try  to  force  an 
adaptation  as  described  earlier.  In  this  case,  a  new  way  to  teach  the 
tasks  involved  would  have  to  be  found,  or  the  tasks  deleted  from  the 
program. 

The  same  holds  for  training  and  job  aids  of  low  quality  (items  7  and 
8)  unless  they  can  be  safely  adapted  as  described  in  Section  III.  If  these 
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aids  are  not  sufficient  in  number,  the  discussion  in  that  section  regarding 
large  trainee/equipment  ratios  is  pertinent. 

The  issue  of  a  facilitating  environment  and  adequate  training  facili¬ 
ties  (items  5  and  9)  reduces  ultimately  to  a  question  of  priority:  How 
important  is  training  compared  with  other  responsibilities  of  the  training 
organization?  If  It  is  important,  provide  the  space  and  physical  environment 
needed.  The  principle  barrier  to  providing  adequate  space  and  facilities 
is  a  belief  that  training  can  be  accomplished  under  almost  any  conditions. 
Rarely  is  this  assumption  justified.  Not  only  does  training  efficiency 
suffer  when  facilities  are  inadequate,  but  bad  morale  is  likely.  As 
explained  later,  negative  attitudes  resulting  from  bad  morale  can  undermine 
any  training  program. 

Items  4,  6,  and  10-12  can  also  be  handled  readily,  given  a  resolve  to 
do  so.  And  as  is  so  often  the  case,  it  Is  better  to  prevent  these  dif¬ 
ficulties  than  to  deal  with  their  effects.  Prevention  can  be  accomplished 
most  easily  by  creating  a  mechanism  for  doing  so,  and  designating  someone 
to  see  that  it  functions  properly.  Task  documentation  can  be  made 
available;  the  maintenance  status  of  training  equipment  can  be  monitored, 
and  information  regarding  maintenance  can  be  communicated  to  a  designated 
person  who  is  responsible  for  timely  corrective  actions.  Monitoring 
training  quality  (item  10)  calls  for  instituting  a  formal  quality  control 
program  for  identifying  training  deficiencies,  for  providing  mechanisms  for 
their  correction,  and  for  ensuring  administrative  cognizance  of  training 
practices  and  effectiveness. 

Actions  such  as  these  would  be  visible  indications  of  administrative 
Interest  In  and  commitment  to  training.  They  would  not  only  provide  the 
support  needed,  but  they  would  go  far  toward  ensuring  positive  attitudes  of 
trainees  and  training  personnel  that  are  critical  to  quality  training. 
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INSTRUCTOR  CHARACTERISTICS  AND  QUALIFICATIONS 

Three  common  problems  related  to  Instructor  characteristics  and  quali¬ 
fications  are: 

1.  Instructors  may  have  undesirable  attitudes  toward  training  and 
their  roles  in  a  program. 

2.  They  may  not  have  the  skill  proficiency  required  for  the  program  as 
designed. 

3.  They  may  not  be  adequately  trained  to  teach. 

The  first  two  problems  are  largely  matters  of  instructor  selection  or  of 
the  availability  of  appropriately  qualified  personnel  from  whom  instructors 
may  be  selected.  However,  undesirable  attitudes  may  develop  after  instruc¬ 
tional  duties  begin  (see  later  discussion  of  Management  of  Attitudes).  The 
third  problem  can  be  resolved  by  providing  the  necessary  training. 

Caro,  Shelnutt,  and  Spears1  examined  problems  of  selecting,  training, 
and  managing  instructors  for  aircrew  training  programs.  Their  findings  and 
recommendations  hold  for  any  instructors,  however.  The  guides  that  follow 
are  adapted  from  their  report. 

The  selection  of  instructors  should  be  based  on  their  desire  and  abil¬ 
ity  to  do  the  job.  Ideally,  they  should  have  sought  the  assignment.  They 
should  have  demonstrated  above  average  performance  in  past  assignments,  and 
they  should  be  mature,  stable  Individuals  who  can  work  well  with  other  people. 

Instructors  should  be  fully  qualified  in  the  skills  they  are  to  teach. 
Their  effectiveness  would  generally  be  enhanced  if  they  had  also  had  opera¬ 
tional  experience  with  the  tasks  to  be  taught,  for  they  could  then  provide 
a  realistic  atmosphere  during  training  that  can  help  trainees  understand 
the  Importance  of  what  they  are  learning. 

*Caro,  P.W.,  Shelnutt,  J.B.,  and  Spears,  W.D.  Utilization  of  Aircrew 
Training  Devices  (Tech.  Rep.  TR  80-01).  Pensacola,  Ft!  Seville  Research 
Corporation,  February  1980. 
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Before  they  begin  teaching.  Instructors  should  go  through  comprehen¬ 
sive,  formally  structured  training  to  prepare  them  to  teach.  Following 
training,  the  Instructor  should  be  able  to  demonstrate  a  practical 
knowledge  of  instructional  technology  such  as  that  Implied  In  Sections  II 
and  III.  Also,  his  training  should  remove  any  deficiencies  In  his  own  per¬ 
formance  of  the  skills  to  be  taught,  and  update  his  skills  If  task  require¬ 
ments  have  changed  since  he  last  performed  the  tasks. 

The  instructor  trainee  should  practice  teaching  the  skills  of  concern, 
with  supervision  and  critiques  by  a  qualified  Instructor.  While  doing  so, 
common  student  problems  should  be  reviewed  and  the  adaption  of  training 
techniques  discussed  with  the  supervisor. 

During  his  tenure  as  an  instructor,  the  instructor  should  receive 
continuation  training  as  needed  to  refresh,  refine,  update,  and  expand  his 
instructional  capabilities.  Scheduled  periodic  refresher  training  will 
guard  against  tendencies  to  become  slipshod  in  teaching,  and  to  depart  from 
standard  instructional  procedures  and  criteria  for  evaluating  student  per¬ 
formance.  Changes  In  skill  performance  requirements  and  newly  validated 
teaching  techniques,  of  course,  should  be  reviewed  when  they  appear. 

Instructional  duties  of  Instructors  should  be  given  priority. 
Additional  non- instructional  responsibilities  should  not  interfere  with  the 
time  and  effort  required  to  teach,  nor  make  instruction  unduly  burdensome. 
Moreover,  evaluation  of  instructional  proficiency  should  be  weighted 
heavily  in  overall  instructor  proficiency  reports. 

MANAGEMENT  OF  ATTITUDES 

HPGI  listed  four  conditions  and  types  of  observations  that  indicate  a 
problem  with  attitudes  toward  training: 
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1.  Negative  consequences  for  good  test  performance  (e.g.,  peer 
disapproval,  undesirable  post-training  assignments). 

2.  Absence  of  consequences  for  good  test  performance— It  does  not 
matter  how  a  trainee  performs. 

3.  Trainee  absenteeism,  tardiness,  careless  work,  negative  comments, 
etc. 

4.  Instructor  absenteeism,  tardiness,  careless  work,  negative  com¬ 
ments,  etc. 

Poor  attitudes  result  in  poor  training.  For  example,  peer  pressure 
that  prevents  a  trainee  from  performing  well  on  a  test  will  also  prevent  him 
from  learning  the  skills  in  the  first  place.  The  same  can  be  said  for  unde¬ 
sirable  post-training  assignments.  Or,  if  it  does  not  matter  how  one  per¬ 
forms  on  a  test,  then  it  does  not  really  matter  what  one  learns  (see 
discussions  of  uses  of  feedback  in  Section  II). 

There  should  be  no  question  that  unfavorable  attitudes  result  in  less 
skill  development  than  would  be  the  case  if  favorable  attitudes  prevailed. 
For  this  reason,  negative  attitudes  among  students  and  instructors  should 
be  a  major  administrative  concern. 

Successful  management  of  attitudes  requires  a  basic  understanding  of 
how  they  are  formed  and  maintained.  Attitudes  are  learned,  but  unlike 
skill  learning,  a  favorable  or  unfavorable  attitude  toward  training  or  a 
job  can  be  learned  thoroughly  by  simply  being  exposed  to  a  group  that  holds 
the  attitude.  This  learning  process,  usually  termed  "modeling,"  is  the  most 
profound  basis  for  social  behavior.  Trainees  model  their  peers  and 
instructors.  That  is,  they  adopt  the  beliefs  and  value  systems  of  those 
around  them.  To  do  otherwise  would  lead  to  ridicule  and  ostracism.  There¬ 
fore,  any  attempt  to  foster  favorable  attitudes,  and  to  prevent  unfavorable 
attitudes,  must  focus  on  the  local  training  community  as  a  whole. 
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This  means  that  the  training  environment  Is  the  primary  consideration. 
A  professional  environment  was  stressed  In  Section  II.  It  Is  now  time  to 
examine  factors  that  promote  professionalism  in  trainees  and  instructors. 

First,  the  goals  of  training  must  be  valued  by  the  trainees  and 
Instructors.  The  goals  will  not  be  valued  unless  administrative  personnel 
themselves  value  them.  Furthermore,  administrators  must  demonstrate  that 
the  training  is  important.  It  gets  nowhere--in  fact.  It  is  usually 
counterproductive — to  pay  only  lip  service.  Visible  support  for  training 
and  administrative  actions  with  tangible  results  are  required.  No  matter 
what  he  is  told,  a  trainee  will  not  believe  that  his  efforts  are  worthwhile 
if  he  must  train  with  inadequate  equipment,  if  his  instructors  are  careless 
or  unqualified  for  their  roles,  or  If  his  nontraining  duties  make  con¬ 
centrated  learning  efforts  burdensome.  Instructors  will  not  value  their 
roles  if  adequate  training  resources  are  not  available,  if  nonteaching 
duties  prevent  them  from  giving  instructional  responsibilities  the  full 
attention  they  deserve,  or  if  quality  instruction  is  of  little  or  no  con¬ 
sequence  for  fitness  reports.  In  brief,  if  there  is  no  evidence  that 
anyone  else  cares  about  quality  training,  why  should  a  trainee  or  an 
instructor  care? 

Earlier  discussions  In  this  section  on  Training  Support  and 
Facilitation,  and  Instructor  Characteristics  and  Qualifications,  covered  a 
number  of  points  bearing  on  this  issue,  and  they  need  not  be  repeated  here. 
Those  sections  primarily  addressed  effective  instruction,  but  by  referring 
to  them  the  reader  can  appreciate  the  scope  of  deliberate  efforts  required 
to  demonstrate  that  quality  training  is  valued. 

A  second  consideration  for  fostering  favorable  attitudes  is  the  pre¬ 
vention  of  undesirable  attitudes  in  the  first  place.  Once  formed,  negative 


attitudes  are  difficult  to  change.  Ideally,  trainees  or  instructors  should 
never  be  shanghaied  to  get  them  into  a  program  for  they  will  surely  begin 
with  feelings  of  resentment.  In  practice,  coercion,  at  least  of  a  limited 
sort,  is  often  necessary.  But  negative  effects  on  attitudes  can  be  reduced 
if  trainees  and  instructors  are  selected  according  to  their  likelihood  of 
succeeding  in  their  roles.  Formal  selection  programs  may  require 
considerable  effort,  but  the  payoff  could  make  it  worth  it. 

A  third  consideration  is  to  ensure  success  for  trainees  and  instruc¬ 
tors.  As  emphasized  in  Sections  II  and  III,  the  tasks  to  be  trained  and 
the  pacing  of  the  training  should  be  such  that  trainees  make  steady 
progress.  If  trainees  succeed,  their  instructors  do  too. 

Fourth,  trainee  and  instructor  success  should  be  rewarded.  If 
quality  performance  is  not  recognized,  there  is  no  need  to  strive  for  it. 
Proficiency  advancement  as  discussed  in  Section  III  provides  many  oppor¬ 
tunities  to  reward  trainee  success.  Upon  completing  a  stage  of  training,  a 
trainee  might  be  given  a  choice  of  interesting  optional  tasks  to  practice, 
or  even  free  time,  while  others  catch  up  with  him.  Or,  he  may  progress 
directly  to  the  next  training  task,  with  the  privileges  provided  at  the  end 
of  training.  (It  is  hard  for  peer  pressure  to  interfere  with  progress  when 
a  desired  reward  is  received  for  reaching  proficiency.  In  fact,  negative 
peer  pressures  would  probably  not  even  develop.) 

There  are  many  ways  to  reward  both  trainees  and  instructors  for  suc¬ 
cessful  performance.  In  many  situations,  simple  published  recognition  by 
the  training  organization  Is  adequate.  But  if  no  tangible  rewards  are 
offered,  there  Is  always  the  danger  that  official  recognition  will  be 
viewed  as  a  "gold  star"  for  reading  the  Sunday  school  lesson. 

Fifth,  boredom  should  be  avoided  to  the  extent  practical.  Section  III 
discussed  ways  to  prevent  or  alleviate  this  problem  during  practice. 
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Similar  points  can  be  made  concerning  all  activities,  official  or  personal. 
The  rule  to  keep  in  mind  is,  if  a  trainee  (or  instructor)  is  idle  for  long 
periods,  let  it  be  at  his  own  choice  and  on  his  own  time.  That  Is,  he  can 
do  what  he  wishes,  including  nothing,  when  he  is  not  on  duty;  but  when  he 
is  on  duty  he  must  work  at  his  job.  The  pace  of  work  should  be  reasonable, 
of  course,  and  with  rest  periods.  The  thing  to  avoid  is  opportunities  to 
let  socializing,  daydreaming,  and  other  unrelated  pursuits  distract  atten¬ 
tion  from  training.  A  coffee  break  should  be  a  clear  break  in  the  routine, 
not  combined  with  a  serious  training  activity. 

Finally,  there  should  be  identifiable  advocates  of  the  training 
program  and  the  job  for  which  trainees  are  being  prepared.  One  or  more 
respected  persons  should  demonstrate  through  their  own  enthusiasm  that  the 
training  program  and  its  goals  are  important.  Enthusiastic  instructors 
fill  this  role  naturally.  But  a  formally  designated  spokesman  can  increase 
the  instructors'  impact.  Bulletin  boards  are  used  effectively  by  most 
training  organizations  to  promote  programs  and  goals.  Personal  contacts 
with  a  respected  advocate  could  be  even  more  effective. 

PROBLEMS  IN  PERFORMANCE  TESTING 

HPGI  placed  heavy  emphasis  on  problems  related  to  performance  testing. 
If  a  test  is  to  provide  information  about  effectiveness  of  training,  it 
should  be  such  that  the  results  of  effective  training  are  not  obscured  by 
peculiarities  of  the  test.  In  this  light,  the  validity  of  the  performance 
test  is  a  critical  administrative  concern. 

HPGI  listed  16  problems  related  to  performance  tests: 

1.  Mismatch  between  training  and  test  conditions. 

2.  Performance  criteria  unreasonably  high. 

3.  Test  criteria  very  much  higher  than  training  criteria. 


61 


k 


K 


ci 


4.  Trainees  not  tested  on  same  content  taught  during  training. 

5.  Equipment  used  In  test  different  from  that  used  In  training. 

6.  Job  aids  used  during  training  but  not  on  test. 

7.  Pass/fail  criteria  not  clear. 

8.  Negative  consequences  for  quality  performance. 

9.  No  consequences  for  quality  performance. 

10.  Poor  trainee  attitude  during  test. 

11.  Boredom  due  to  tedium  of  test  tasks. 

12.  Poor  test  administration. 

13.  Use  of  inconsistent  standards  for  scoring  performance. 

14.  Poor  directions  provided  during  testing. 

15.  Excessive  pressure  during  testing. 

16.  Performance  on  test  (i.e.,  test  results)  "contaminated"  by 
unauthorized  intervention  of  test  personnel. 

The  first  11  have  been  discussed  previously.  Parts  of  these 
discussions  will  be  reviewed  briefly  to  emphasize  a  few  points. 

The  need  for  a  systematic  procedure  for  designing  training  programs 
was  discussed  at  the  beginning  of  this  section.  Such  a  procedure,  extended 
to  the  final  test,  would  take  care  of  problems  identified  in  the  first 
seven  items.  However,  the  person  who  uses  these  seven  items  as  guides  for 
determining  the  appropriateness  of  a  test  must  be  forewarned  that  cir¬ 
cumstances  may  not  be  as  they  appear.  For  example,  it  is  common--and 
desirable  when  feasible— for  a  final  test  to  sample  operational  conditions 
and  performance  requirements  whether  or  not  they  characterized  training. 
Efficient  training  usually  requires  special,  contrived  conditions. 
Therefore,  a  systematic  approach  to  designing  a  training  program  may  well 
dictate  that  there  be  differences  in  training  and  testing  conditions.  The 
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purpose  of  the  test  Is  to  determine  the  extent  to  which  skills  learned  In  a 
contrived  training  situation  are  usable  In  operational  settings  which  may 
change  from  day  to  day.  This  Issue  was  discussed  In  Section  III  as  a 
problem  of  realism.  It  was  also  explained  that  a  "dry  run"  under  test  con¬ 
ditions  should  precede  actual  testing. 

Except  for  Item  2— "unreasonably  high"  criteria  are  by  definition 
unreasonable— all  of  the  first  six  Items  require  interpretation.  Test  cri¬ 
teria  may  be  higher  because  they  could  not  be  used  as  such  during  training. 
For  example,  imperfectly  simulated  conditions  or  equipment  may  be  used 
during  training  that  make  operational  criteria  inapplicable.  In  such 
situations,  the  question  for  the  analyst  to  ponder  is  not  whether  trainees 
achieve  the  test  criteria  levels,  but  do  they  approach  them  as  closely  as 
can  reasonably  be  expected?  A  similar  point  holds  for  differences  In 
equipment.  A  trainee  may  use  a  simulator  during  training,  but  he  Is  not 
expected  to  go  to  battle  with  It.  Was  what  he  learned  using  a  simulator  (or 
any  other  training  equipment)  applicable  to  performance  with  actual 
equipment? 

Test  content  (item  4)  probably  should  vary  unless  trainees  are  not 
expected  to  do  anything  under  any  conditions  except  as  practiced  during 
training.  This  is  hardly  likely  for  a  tank  driver  or  a  gunner.  Tactical 
terrains  and  maneuvering  requirements,  and  conditions  for  acquiring  targets 
and  estimating  ranges,  are  highly  variable.  Trainees  must  learn  to  perform 
regardless  of  such  variations.  Again,  the  question  is,  does  the  test 
represent  the  same  domain— the  same  general  kind— of  content  that  was 
covered  in  the  training  objectives? 

Regarding  item  6,  a  training  analyst  should  distinguish  between  job 
aids  and  training  aids.  If  the  training  plan  is  to  teach  the  use  of  job 
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aids  every  time  a  task  Is  performed,  the  job  aids  should  be  available  on 
the  test.  If  trainees  are  expected  to  perform  without  them,  their  training 
should  target  this  objective.  The  reader  Is  referred  to  a  discussion  of 
this  topic  In  Section  III. 

Items  8-10  involve  attitudes,  and  these  Items  were  discussed  earlier. 
Item  11  regarding  boredom  was  discussed  under  Attitudes,  and  also  In 
Section  III  as  It  affects  skill  acquisition.  If  boredom  Is  avoided  or 
accommodated  during  training  as  described.  It  will  be  no  problem  during 
testing. 

The  remaining  five  items,  together  with  numbers  2  and  7,  are  clearly 
matters  of  inadequate  testing.  As  stated,  2  and  7  can  be  resolved  through 
a  systematic  design  for  the  training  program.  Items  12-16  call  for 
specific  administrative  attention. 

Requirements  for  adequate  testing  all  flow  from  one  principle:  Dif¬ 
ferences  in  test  performance  should  reflect  differences  among  Individuals 
in  the  characteristics  of  interest.  Such  an  interpretation  is  possible 
only  to  the  extent  the  test  is  valid  and  its  administration  is  standard¬ 
ized.  Validity  was  the  concern  of  the  discussion  Immediately  preceding 
and  other  discussions  that  were  cited.  The  focus  now  will  be  on  standard¬ 
ization  of  testing  procedures. 

In  brief,  standardization  means  that  every  trainee  is  tested  under 
comparable  conditions  and  requirements.  The  directions  they  receive  are 
accurate,  complete,  and  understood  in  the  same  way  by  all  trainees.  For 
any  given  test,  the  same  tasks  are  always  presented  in  the  same  way  and  in 
the  same  sequence,  and  standardized  time  limits  are  used.  Any  intervention 
by  testing  personnel,  whether  helpful  or  otherwise,  follows  a  consistent 
pattern.  Its  nature  Is  specified  beforehand,  along  with  the  circumstances 
under  which  Interventions  are  to  occur. 
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The  problem  of  Inconsistent  scoring  of  performance  (Item  13)  Is  very 
common.  It  Is  also  very  serious.  Not  only  do  different  scorers  often 
disagree  with  each  other,  a  given  scorer  can  be  Inconsistent  with  himself. 

The  only  way  to  avoid  the  problem  Is  to  have  clear-cut  scoring  criteria 
(preferably  objective  Indicators),  and  train  personnel  to  use  them  In  the 
same  way.  If  two  scorers  working  Independently  score  the  same  performance 
and  get  different  results,  either  the  standards  are  unclear  or  the  scorers 
have  not  been  adequately  trained. 

A  frequent  cause  of  scoring  inconsistences  Is  a  dependence  on  an 
overall  rating  of  performance.  A  trainee  may  do,  say,  ten  things,  six  of 
them  correctly,  two  Incorrectly,  and  two  of  questionable  adequacy.  How 
much  can  an  observer  remember  about  these  details  when  he  assigns  an 
overall  rating?  Is  he  likely  to  remember  performance  on  every  step  and 
always  weight  each  step  the  same  amount?  Just  raising  these  questions 
reveals  that  a  more  systematic  procedure  is  needed.  It  would  be  better  to 
use  a  checklist  and  identify  each  correct  and  incorrect  action,  each  cue 
noticed  and  missed,  etc.  With  a  record  of  the  actual  performance, 
evaluations  could  be  made  later  according  to  a  standardized  scheme. 

Item  15  refers  to  "excessive"  pressures  that  prevent  a  trainee  from 
demonstrating  what  he  can  do.  Undue  pressures  can  be  difficult  to  deal 
with,  so  some  explanation  of  what  Is  involved  is  in  order.  First, 
conscientious  trainees  will  feel  some  pressure  any  time  they  take  a  formal 
test,  and  the  amount  of  pressure  will  relate  to  their  confidence  in  their 
ability  to  perform.  If  many  trainees  exhibit  an  unusual  nervousness, 
however,  it  means  that  the  test  is  unnecessarily  threatening  to  them. 

(As  a  check  on  such  an  inference,  one  might  determine  if  trainees  had  shown 
proficiency  during  training  on  the  skills  they  do  poorly  on  during  the  test.) 


Anxiety  regarding  tests  arises  In  many  ways,  most  often  from  comments  by 
Instructors  and  good  natured  harassment  by  other  trainees  who  have  already 
"been  through  the  mill."  Tensions  can  be  eased  If  Instructional  personnel 
exhibit  a  general  confidence  In  graduates  of  the  training  program,  pointing 
to  success  rates  of  previous  graduates. 

Pressures  also  arise  when  time  limits  for  performance  during  tests  are 
more  strict  than  the  trainees  are  accustomed  to,  or  when  new  applications 
of  skills  will  be  necessary.  These  pressures  can  be  reduced  by  Imposing 
the  same  time  limits,  and  requiring  new  applications,  during  later  stages 
of  training. 

The  task  for  a  training  analyst  is  to  determine  when  pressures  are 
"excessive"  and  when  they  are  not.  If  a  trainee  cannot  perform  under 
reasonable  pressures  for  whatever  reason,  his  training  has  not  been 
completely  successful.  If  pressures  appear  unreasonable,  however,  then  the 
test  cannot  reveal  what  It  Is  supposed  to.  In  this  case,  corrective 
actions  are  needed. 

SUMMARY 

This  section  discussed  training  deficiencies  that  arise  because  of 
administrative  practices  and  neglect.  Forty-five  problems  or  indications  of 
problems  were  addressed,  some  more  than  once.  They  were  grouped  under  five 
heads:  Indications  of  Need  to  Redesign  Training  Programs;  Training  Support 
and  Facilitation;  Instructor  Characteristics  and  Qualifications;  Management 
of  Attitudes;  and  Problems  In  Performance  Testing.  The  focus  for  each 
discussion  was  the  need  for  administrative  resolve  and  action  to  prevent 
problems  from  arising  when  possible,  and  to  take  prompt  corrective  steps 
when  they  do  occur. 
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